[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:19:51 dist-test-slave-pd0t krb5kdc[4913](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:19:51 dist-test-slave-pd0t krb5kdc[4913](info): set up 2 sockets
Sep 26 06:19:51 dist-test-slave-pd0t krb5kdc[4913](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260926 06:19:52.659001 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.001s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:19:52 dist-test-slave-pd0t krb5kdc[4913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:19:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:19:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.202.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
I20260926 06:19:52.773619 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:37271
--webserver_interface=127.4.202.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:45459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:37271
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:19:52.850157 4926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:19:52.850360 4926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:19:52.850392 4926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:19:52.851490 4926 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:19:52.851552 4926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:19:52.851575 4926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:19:52.851610 4926 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:19:52.851641 4926 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:19:52.853029 4926 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:45459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:37271
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:37271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:19:52.853324 4926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:19:52.853557 4926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:19:52.858480 4934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:19:52.858465 4931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:19:52.858507 4926 server_base.cc:1034] running on GCE node
W20260926 06:19:52.858456 4932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:19:52.858891 4926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:19:52.859155 4926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:19:52.860294 4926 hybrid_clock.cc:648] HybridClock initialized: now 1790403592860270 us; error 41 us; skew 500 ppm
Sep 26 06:19:52 dist-test-slave-pd0t krb5kdc[4913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403592, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:19:52.862828 4926 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:19:52.863258 4926 webserver.cc:460] Webserver started at http://127.4.202.190:42465/ using document root <none> and password file <none>
I20260926 06:19:52.863477 4926 fs_manager.cc:359] Metadata directory not provided
I20260926 06:19:52.863519 4926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:19:52.863658 4926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:19:52.864636 4926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "9c2734b833374e73b20a700362ed7a63"
format_stamp: "Formatted at 2026-09-26 06:19:52 on dist-test-slave-pd0t"
server_key: "aeb2280fda101510ae5a847d4b0edff0"
server_key_iv: "0a13ef0c5c4aa153952d6a7722a1719c"
server_key_version: "encryptionkey@0"
I20260926 06:19:52.864988 4926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "9c2734b833374e73b20a700362ed7a63"
format_stamp: "Formatted at 2026-09-26 06:19:52 on dist-test-slave-pd0t"
server_key: "aeb2280fda101510ae5a847d4b0edff0"
server_key_iv: "0a13ef0c5c4aa153952d6a7722a1719c"
server_key_version: "encryptionkey@0"
I20260926 06:19:52.866346 4926 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260926 06:19:52.867121 4941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:52.867300 4926 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:52.867389 4926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "9c2734b833374e73b20a700362ed7a63"
format_stamp: "Formatted at 2026-09-26 06:19:52 on dist-test-slave-pd0t"
server_key: "aeb2280fda101510ae5a847d4b0edff0"
server_key_iv: "0a13ef0c5c4aa153952d6a7722a1719c"
server_key_version: "encryptionkey@0"
I20260926 06:19:52.867483 4926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:19:52.881037 4926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:19:52.882777 4926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:19:52.882936 4926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:19:52.886533 4926 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:37271
I20260926 06:19:52.886564 4993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:37271 every 8 connection(s)
I20260926 06:19:52.886924 4926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:19:52.887705 4994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:19:52.887964 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 4926
I20260926 06:19:52.888063 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:19:52.888638 4906 external_mini_cluster.cc:1428] Setting key 84980225f03a3f3a8470ae576124f5da
I20260926 06:19:52.890228 4994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c2734b833374e73b20a700362ed7a63: Bootstrap starting.
I20260926 06:19:52.890895 4994 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c2734b833374e73b20a700362ed7a63: Neither blocks nor log segments found. Creating new log.
I20260926 06:19:52.891220 4994 log.cc:824] T 00000000000000000000000000000000 P 9c2734b833374e73b20a700362ed7a63: Log is configured to *not* fsync() on all Append() calls
I20260926 06:19:52.892030 4994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c2734b833374e73b20a700362ed7a63: No bootstrap required, opened a new log
I20260926 06:19:52.894244 4994 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c2734b833374e73b20a700362ed7a63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c2734b833374e73b20a700362ed7a63" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 37271 } }
I20260926 06:19:52.894404 4994 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c2734b833374e73b20a700362ed7a63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:19:52.894445 4994 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c2734b833374e73b20a700362ed7a63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2734b833374e73b20a700362ed7a63, State: Initialized, Role: FOLLOWER
I20260926 06:19:52.894562 4994 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c2734b833374e73b20a700362ed7a63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c2734b833374e73b20a700362ed7a63" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 37271 } }
I20260926 06:19:52.894690 4994 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c2734b833374e73b20a700362ed7a63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:19:52.894762 4994 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c2734b833374e73b20a700362ed7a63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:19:52.894810 4994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c2734b833374e73b20a700362ed7a63 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:19:52.895364 4994 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c2734b833374e73b20a700362ed7a63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c2734b833374e73b20a700362ed7a63" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 37271 } }
I20260926 06:19:52.895447 4994 leader_election.cc:302] T 00000000000000000000000000000000 P 9c2734b833374e73b20a700362ed7a63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c2734b833374e73b20a700362ed7a63; no voters:
I20260926 06:19:52.895639 4994 leader_election.cc:288] T 00000000000000000000000000000000 P 9c2734b833374e73b20a700362ed7a63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:19:52.895663 4999 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c2734b833374e73b20a700362ed7a63 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:19:52.895764 4999 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c2734b833374e73b20a700362ed7a63 [term 1 LEADER]: Becoming Leader. State: Replica: 9c2734b833374e73b20a700362ed7a63, State: Running, Role: LEADER
I20260926 06:19:52.895843 4999 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c2734b833374e73b20a700362ed7a63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c2734b833374e73b20a700362ed7a63" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 37271 } }
I20260926 06:19:52.895887 4994 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c2734b833374e73b20a700362ed7a63 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:19:52.896145 4999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c2734b833374e73b20a700362ed7a63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c2734b833374e73b20a700362ed7a63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c2734b833374e73b20a700362ed7a63" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 37271 } } }
I20260926 06:19:52.896227 4999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c2734b833374e73b20a700362ed7a63 [sys.catalog]: This master's current role is: LEADER
I20260926 06:19:52.896353 4999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c2734b833374e73b20a700362ed7a63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c2734b833374e73b20a700362ed7a63. Latest consensus state: current_term: 1 leader_uuid: "9c2734b833374e73b20a700362ed7a63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c2734b833374e73b20a700362ed7a63" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 37271 } } }
I20260926 06:19:52.896420 4999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c2734b833374e73b20a700362ed7a63 [sys.catalog]: This master's current role is: LEADER
W20260926 06:19:52.897608 5015 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9c2734b833374e73b20a700362ed7a63: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:19:52.897689 5015 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:19:52.897739 5004 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:19:52.897894 5004 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:19:52.899387 5004 catalog_manager.cc:1347] Generated new cluster ID: d98a1452843e441cba9912e07fd1cb0c
I20260926 06:19:52.899438 5004 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:19:52 dist-test-slave-pd0t krb5kdc[4913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:19:52.906307 4997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:52.889411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41254 (local address 127.4.202.190:37271)
0926 06:19:52.889686 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:52.889693 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:52.889800 (+ 107us) server_negotiation.cc:407] Connection header received
0926 06:19:52.892475 (+ 2675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:52.892481 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:52.892522 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:52.892655 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:19:52.894706 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:52.896278 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:52.897578 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:52.898128 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:52.898250 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:52.902897 (+ 4647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:19:52.902900 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:19:52.902900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:19:52.902910 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:19:52.904550 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:52.904969 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:52.904972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:52.904972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:52.905055 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:52.905312 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:52.905313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:52.905313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:52.905526 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:19:52.905701 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:52.905966 (+ 265us) server_negotiation.cc:299] Negotiation successful
0926 06:19:52.906097 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":67,"threads_started":1}
I20260926 06:19:52.936929 5004 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:19:52.937425 5004 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:19:52.940325 5004 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c2734b833374e73b20a700362ed7a63: Generated new TSK 0
I20260926 06:19:52.940480 5004 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.202.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
I20260926 06:19:52.973872 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.129:0
--local_ip_for_outbound_sockets=127.4.202.129
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:37271
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:45459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:19:53.052294 5020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:19:53.052471 5020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:19:53.052493 5020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:19:53.053588 5020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:19:53.053634 5020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:19:53.053660 5020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.129
I20260926 06:19:53.055087 5020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:45459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:37271
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.129
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:19:53.055294 5020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:19:53.055522 5020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:19:53.060284 5028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:19:53.060402 5026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:19:53.060284 5025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:19:53.060513 5020 server_base.cc:1034] running on GCE node
I20260926 06:19:53.060708 5020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:19:53.060935 5020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:19:53.062074 5020 hybrid_clock.cc:648] HybridClock initialized: now 1790403593062069 us; error 29 us; skew 500 ppm
Sep 26 06:19:53 dist-test-slave-pd0t krb5kdc[4913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403593, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:19:53.064424 5020 init.cc:377] Logged in from keytab as kudu/127.4.202.129@KRBTEST.COM (short username kudu)
I20260926 06:19:53.064845 5020 webserver.cc:460] Webserver started at http://127.4.202.129:40819/ using document root <none> and password file <none>
I20260926 06:19:53.065054 5020 fs_manager.cc:359] Metadata directory not provided
I20260926 06:19:53.065119 5020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:19:53.065289 5020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:19:53.066262 5020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-0/data/instance:
uuid: "70146e2fb6d24c22a6d88b0c70d0b051"
format_stamp: "Formatted at 2026-09-26 06:19:53 on dist-test-slave-pd0t"
server_key: "75d5c10e727b3695640e86aec22d71f1"
server_key_iv: "084beeda40720d26926ca5afbc75bc02"
server_key_version: "encryptionkey@0"
I20260926 06:19:53.066705 5020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance:
uuid: "70146e2fb6d24c22a6d88b0c70d0b051"
format_stamp: "Formatted at 2026-09-26 06:19:53 on dist-test-slave-pd0t"
server_key: "75d5c10e727b3695640e86aec22d71f1"
server_key_iv: "084beeda40720d26926ca5afbc75bc02"
server_key_version: "encryptionkey@0"
I20260926 06:19:53.067993 5020 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:19:53.068805 5035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:53.068987 5020 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:19:53.069079 5020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-0/wal
uuid: "70146e2fb6d24c22a6d88b0c70d0b051"
format_stamp: "Formatted at 2026-09-26 06:19:53 on dist-test-slave-pd0t"
server_key: "75d5c10e727b3695640e86aec22d71f1"
server_key_iv: "084beeda40720d26926ca5afbc75bc02"
server_key_version: "encryptionkey@0"
I20260926 06:19:53.069146 5020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:19:53.091547 5020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:19:53.093401 5020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:19:53.093562 5020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:19:53.093796 5020 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:19:53.094095 5020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:19:53.094156 5020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:53.094188 5020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:19:53.094215 5020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:53.101214 5020 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.129:45755
I20260926 06:19:53.101256 5148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.129:45755 every 8 connection(s)
I20260926 06:19:53.101553 5020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
Sep 26 06:19:53 dist-test-slave-pd0t krb5kdc[4913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403593, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:19:53.108444 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 5020
I20260926 06:19:53.108544 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance
I20260926 06:19:53.108750 4906 external_mini_cluster.cc:1428] Setting key 5fffeb2458511cbf4e24ac84e8075bdb
I20260926 06:19:53.111292 4997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:53.102248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:48605 (local address 127.4.202.190:37271)
0926 06:19:53.102327 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:53.102330 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:53.102498 (+ 168us) server_negotiation.cc:407] Connection header received
0926 06:19:53.103097 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:53.103098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:53.103110 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:53.103167 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:19:53.104075 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:53.104551 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:53.105510 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:53.105750 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:53.105836 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:53.107854 (+ 2018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:19:53.107856 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:19:53.107856 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:19:53.107864 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:19:53.109600 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:53.110134 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:53.110135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:53.110135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:53.110185 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:53.110482 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:53.110483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:53.110483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:53.110613 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:19:53.110707 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:53.111052 (+ 345us) server_negotiation.cc:299] Negotiation successful
0926 06:19:53.111162 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 06:19:53.111398 5151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:53.102213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37271 (local address 127.4.202.129:48605)
0926 06:19:53.102388 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:53.102420 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:53.102987 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:53.103248 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:53.103252 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:53.103788 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:19:53.103962 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:53.103966 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:53.104651 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:53.104653 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:53.105415 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:53.105417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:53.105846 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:53.105847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:53.105969 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:53.106413 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:19:53.106421 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:19:53.107755 (+ 1334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:19:53.109676 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:53.109682 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:53.109684 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:53.110043 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:53.110248 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:53.110249 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:53.110250 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:53.110421 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:53.110704 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:19:53.110728 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:19:53.110890 (+ 162us) client_negotiation.cc:769] Sending connection context
0926 06:19:53.111015 (+ 125us) client_negotiation.cc:240] Negotiation successful
0926 06:19:53.111204 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":79,"threads_started":1}
I20260926 06:19:53.111897 5149 heartbeater.cc:347] Connected to a master server at 127.4.202.190:37271
I20260926 06:19:53.112004 5149 heartbeater.cc:464] Registering TS with master...
I20260926 06:19:53.112169 5149 heartbeater.cc:511] Master 127.4.202.190:37271 requested a full tablet report, sending...
I20260926 06:19:53.112673 4958 ts_manager.cc:194] Registered new tserver with Master: 70146e2fb6d24c22a6d88b0c70d0b051 (127.4.202.129:45755)
I20260926 06:19:53.113207 4958 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.129@KRBTEST.COM'} at 127.4.202.129:48605
WARNING: no policy specified for kudu/127.4.202.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
I20260926 06:19:53.132695 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.130:0
--local_ip_for_outbound_sockets=127.4.202.130
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:37271
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:45459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:19:53.211650 5154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:19:53.211833 5154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:19:53.211858 5154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:19:53.212999 5154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:19:53.213042 5154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:19:53.213068 5154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.130
I20260926 06:19:53.214465 5154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:45459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:37271
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.130
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:19:53.214747 5154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:19:53.214962 5154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:19:53.219974 5159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:19:53.219995 5162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:19:53.220009 5160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:19:53.220288 5154 server_base.cc:1034] running on GCE node
I20260926 06:19:53.220501 5154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:19:53.220688 5154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:19:53.221824 5154 hybrid_clock.cc:648] HybridClock initialized: now 1790403593221809 us; error 33 us; skew 500 ppm
Sep 26 06:19:53 dist-test-slave-pd0t krb5kdc[4913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403593, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:19:53.224272 5154 init.cc:377] Logged in from keytab as kudu/127.4.202.130@KRBTEST.COM (short username kudu)
I20260926 06:19:53.224696 5154 webserver.cc:460] Webserver started at http://127.4.202.130:37963/ using document root <none> and password file <none>
I20260926 06:19:53.224920 5154 fs_manager.cc:359] Metadata directory not provided
I20260926 06:19:53.224987 5154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:19:53.225147 5154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:19:53.226172 5154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-1/data/instance:
uuid: "55f30581748d4e1d87e4eabe2482342c"
format_stamp: "Formatted at 2026-09-26 06:19:53 on dist-test-slave-pd0t"
server_key: "a36ec515daa60fddc6797a4c97981178"
server_key_iv: "7da47eaee20ecb79c75262a50810a752"
server_key_version: "encryptionkey@0"
I20260926 06:19:53.226537 5154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance:
uuid: "55f30581748d4e1d87e4eabe2482342c"
format_stamp: "Formatted at 2026-09-26 06:19:53 on dist-test-slave-pd0t"
server_key: "a36ec515daa60fddc6797a4c97981178"
server_key_iv: "7da47eaee20ecb79c75262a50810a752"
server_key_version: "encryptionkey@0"
I20260926 06:19:53.227856 5154 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:19:53.228659 5169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:53.228850 5154 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:19:53.228956 5154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-1/wal
uuid: "55f30581748d4e1d87e4eabe2482342c"
format_stamp: "Formatted at 2026-09-26 06:19:53 on dist-test-slave-pd0t"
server_key: "a36ec515daa60fddc6797a4c97981178"
server_key_iv: "7da47eaee20ecb79c75262a50810a752"
server_key_version: "encryptionkey@0"
I20260926 06:19:53.229045 5154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:19:53.244616 5154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:19:53.246757 5154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:19:53.246913 5154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:19:53.247133 5154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:19:53.247416 5154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:19:53.247465 5154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:53.247498 5154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:19:53.247524 5154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:53.253470 5154 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.130:45171
I20260926 06:19:53.253540 5282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.130:45171 every 8 connection(s)
I20260926 06:19:53.253847 5154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
I20260926 06:19:53.257354 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 5154
I20260926 06:19:53.257441 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance
I20260926 06:19:53.257582 4906 external_mini_cluster.cc:1428] Setting key 8944ef3ff08c25f7ec535066bdb23b52
Sep 26 06:19:53 dist-test-slave-pd0t krb5kdc[4913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403593, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:19:53.263000 4997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:53.254472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:43825 (local address 127.4.202.190:37271)
0926 06:19:53.254546 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:53.254548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:53.254779 (+ 231us) server_negotiation.cc:407] Connection header received
0926 06:19:53.255206 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:53.255207 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:53.255216 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:53.255264 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:19:53.256068 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:53.256571 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:53.257874 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:53.258114 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:53.258202 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:53.260186 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:19:53.260188 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:19:53.260189 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:19:53.260196 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:19:53.261577 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:53.261974 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:53.261975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:53.261976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:53.262035 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:53.262293 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:53.262294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:53.262294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:53.262405 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:19:53.262479 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:53.262832 (+ 353us) server_negotiation.cc:299] Negotiation successful
0926 06:19:53.262915 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 06:19:53.263178 5285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:53.254439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37271 (local address 127.4.202.130:43825)
0926 06:19:53.254670 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:53.254717 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:53.255132 (+ 415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:53.255312 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:53.255315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:53.255826 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:19:53.255993 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:53.255997 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:53.256690 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:53.256690 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:53.257783 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:53.257785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:53.258211 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:53.258213 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:53.258334 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:53.258825 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:19:53.258833 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:19:53.260090 (+ 1257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:19:53.261676 (+ 1586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:53.261682 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:53.261684 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:53.261896 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:53.262110 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:53.262111 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:53.262111 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:53.262218 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:53.262470 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:19:53.262476 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:19:53.262703 (+ 227us) client_negotiation.cc:769] Sending connection context
0926 06:19:53.262811 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 06:19:53.262958 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":84,"threads_started":1}
I20260926 06:19:53.263553 5283 heartbeater.cc:347] Connected to a master server at 127.4.202.190:37271
I20260926 06:19:53.263646 5283 heartbeater.cc:464] Registering TS with master...
I20260926 06:19:53.263809 5283 heartbeater.cc:511] Master 127.4.202.190:37271 requested a full tablet report, sending...
I20260926 06:19:53.264183 4958 ts_manager.cc:194] Registered new tserver with Master: 55f30581748d4e1d87e4eabe2482342c (127.4.202.130:45171)
I20260926 06:19:53.264559 4958 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.130@KRBTEST.COM'} at 127.4.202.130:43825
WARNING: no policy specified for kudu/127.4.202.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
I20260926 06:19:53.282008 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.131:0
--local_ip_for_outbound_sockets=127.4.202.131
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:37271
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:45459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:19:53.361264 5288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:19:53.361586 5288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:19:53.361660 5288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:19:53.363245 5288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:19:53.363286 5288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:19:53.363312 5288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.131
I20260926 06:19:53.365163 5288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:45459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:37271
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.131
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:19:53.365412 5288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:19:53.365650 5288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:19:53.370867 5294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:19:53.370848 5296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:19:53.371021 5288 server_base.cc:1034] running on GCE node
W20260926 06:19:53.371070 5293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:19:53.371294 5288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:19:53.371502 5288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:19:53.372633 5288 hybrid_clock.cc:648] HybridClock initialized: now 1790403593372618 us; error 38 us; skew 500 ppm
Sep 26 06:19:53 dist-test-slave-pd0t krb5kdc[4913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403593, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:19:53.375006 5288 init.cc:377] Logged in from keytab as kudu/127.4.202.131@KRBTEST.COM (short username kudu)
I20260926 06:19:53.375461 5288 webserver.cc:460] Webserver started at http://127.4.202.131:38095/ using document root <none> and password file <none>
I20260926 06:19:53.375664 5288 fs_manager.cc:359] Metadata directory not provided
I20260926 06:19:53.375726 5288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:19:53.375927 5288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:19:53.377266 5288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-2/data/instance:
uuid: "3c6e9c10b10d48c7b2e4480f2d200240"
format_stamp: "Formatted at 2026-09-26 06:19:53 on dist-test-slave-pd0t"
server_key: "315f0733cc3b8a3bd134b8ae0ab450f0"
server_key_iv: "ccfeac907e5810f58cc4c536466c5ac8"
server_key_version: "encryptionkey@0"
I20260926 06:19:53.377710 5288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance:
uuid: "3c6e9c10b10d48c7b2e4480f2d200240"
format_stamp: "Formatted at 2026-09-26 06:19:53 on dist-test-slave-pd0t"
server_key: "315f0733cc3b8a3bd134b8ae0ab450f0"
server_key_iv: "ccfeac907e5810f58cc4c536466c5ac8"
server_key_version: "encryptionkey@0"
I20260926 06:19:53.379264 5288 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:19:53.379904 5303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:53.380062 5288 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:19:53.380151 5288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-2/wal
uuid: "3c6e9c10b10d48c7b2e4480f2d200240"
format_stamp: "Formatted at 2026-09-26 06:19:53 on dist-test-slave-pd0t"
server_key: "315f0733cc3b8a3bd134b8ae0ab450f0"
server_key_iv: "ccfeac907e5810f58cc4c536466c5ac8"
server_key_version: "encryptionkey@0"
I20260926 06:19:53.380203 5288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:19:53.387532 5288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:19:53.389168 5288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:19:53.389269 5288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:19:53.389429 5288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:19:53.389642 5288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:19:53.389670 5288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:53.389690 5288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:19:53.389703 5288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:53.396077 5288 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.131:42791
I20260926 06:19:53.396143 5416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.131:42791 every 8 connection(s)
I20260926 06:19:53.396427 5288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
Sep 26 06:19:53 dist-test-slave-pd0t krb5kdc[4913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403593, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:19:53.404825 4997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:53.397097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:45513 (local address 127.4.202.190:37271)
0926 06:19:53.397164 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:53.397166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:53.397420 (+ 254us) server_negotiation.cc:407] Connection header received
0926 06:19:53.397902 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:53.397903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:53.397913 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:53.397961 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:19:53.398891 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:53.399372 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:53.400255 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:53.400429 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:53.400478 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:53.402334 (+ 1856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:19:53.402335 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:19:53.402336 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:19:53.402341 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:19:53.403773 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:53.404104 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:53.404105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:53.404105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:53.404151 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:53.404350 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:53.404350 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:53.404351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:53.404443 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:19:53.404493 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:53.404709 (+ 216us) server_negotiation.cc:299] Negotiation successful
0926 06:19:53.404765 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 06:19:53.404975 5419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:53.397044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37271 (local address 127.4.202.131:45513)
0926 06:19:53.397264 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:53.397330 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:53.397818 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:53.398016 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:53.398019 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:53.398560 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:19:53.398813 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:53.398817 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:53.399448 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:53.399448 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:53.400182 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:53.400183 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:53.400497 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:53.400498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:53.400589 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:53.400984 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:19:53.400992 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:19:53.402256 (+ 1264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:19:53.403841 (+ 1585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:53.403846 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:53.403848 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:53.404040 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:53.404205 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:53.404206 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:53.404206 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:53.404291 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:53.404497 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:19:53.404500 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:19:53.404598 (+ 98us) client_negotiation.cc:769] Sending connection context
0926 06:19:53.404673 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 06:19:53.404775 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":115,"threads_started":1}
I20260926 06:19:53.405469 5417 heartbeater.cc:347] Connected to a master server at 127.4.202.190:37271
I20260926 06:19:53.405542 5417 heartbeater.cc:464] Registering TS with master...
I20260926 06:19:53.405661 5417 heartbeater.cc:511] Master 127.4.202.190:37271 requested a full tablet report, sending...
I20260926 06:19:53.405985 4958 ts_manager.cc:194] Registered new tserver with Master: 3c6e9c10b10d48c7b2e4480f2d200240 (127.4.202.131:42791)
I20260926 06:19:53.406222 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 5288
I20260926 06:19:53.406296 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance
I20260926 06:19:53.406342 4958 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.131@KRBTEST.COM'} at 127.4.202.131:45513
I20260926 06:19:53.406471 4906 external_mini_cluster.cc:1428] Setting key 1b752d19e611a011fb1e9284209e7ada
I20260926 06:19:53.407429 4906 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 06:19:53 dist-test-slave-pd0t krb5kdc[4913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403593, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 06:19:53 dist-test-slave-pd0t krb5kdc[4913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403593, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.202.129@KRBTEST.COM
I20260926 06:19:53.428769 5424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:53.421538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40668 (local address 127.4.202.129:45755)
0926 06:19:53.421759 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:53.421763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:53.421772 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:19:53.421794 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:53.421796 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:53.421810 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:53.421871 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:19:53.422402 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:53.423168 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:53.423857 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:53.424134 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:53.424185 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:53.426288 (+ 2103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:19:53.426295 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:19:53.426295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:19:53.426307 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:19:53.427733 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:53.428042 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:53.428044 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:53.428044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:53.428092 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:53.428320 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:53.428321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:53.428321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:53.428433 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:19:53.428534 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:53.428602 (+ 68us) server_negotiation.cc:299] Negotiation successful
0926 06:19:53.428658 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":146,"threads_started":1}
W20260926 06:19:53.429483 5063 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40668
Sep 26 06:19:53 dist-test-slave-pd0t krb5kdc[4913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403593, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 06:19:53 dist-test-slave-pd0t krb5kdc[4913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403593, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.129@KRBTEST.COM
I20260926 06:19:53.449834 5424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:53.443418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40670 (local address 127.4.202.129:45755)
0926 06:19:53.443475 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:53.443476 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:53.443515 (+ 39us) server_negotiation.cc:407] Connection header received
0926 06:19:53.443586 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:53.443588 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:53.443603 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:53.443651 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:19:53.444321 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:53.444768 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:53.445447 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:53.445627 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:53.445674 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:53.447782 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:19:53.447783 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:19:53.447784 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:19:53.447789 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:19:53.448948 (+ 1159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:53.449234 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:53.449235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:53.449235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:53.449279 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:53.449443 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:53.449444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:53.449444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:53.449522 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:19:53.449605 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:53.449683 (+ 78us) server_negotiation.cc:299] Negotiation successful
0926 06:19:53.449737 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 06:19:53.450335 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 5020
I20260926 06:19:53.454943 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 5154
I20260926 06:19:53.460237 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 5288
I20260926 06:19:53.464063 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 4926
2026-09-26T06:19:53Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1830 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:19:53 dist-test-slave-pd0t krb5kdc[5431](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:19:53 dist-test-slave-pd0t krb5kdc[5431](info): set up 2 sockets
Sep 26 06:19:53 dist-test-slave-pd0t krb5kdc[5431](info): commencing operation
krb5kdc: starting...
W20260926 06:19:54.494000 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:19:54 dist-test-slave-pd0t krb5kdc[5431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:19:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:19:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.202.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
I20260926 06:19:54.604640 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:39553
--webserver_interface=127.4.202.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:35755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:39553
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:19:54.683590 5444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:19:54.683841 5444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:19:54.683877 5444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:19:54.684978 5444 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:19:54.685041 5444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:19:54.685065 5444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:19:54.685096 5444 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:19:54.685129 5444 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:19:54.686535 5444 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:35755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:39553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:39553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:19:54.686832 5444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:19:54.687081 5444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:19:54.691860 5452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:19:54.691854 5450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:19:54.691936 5444 server_base.cc:1034] running on GCE node
W20260926 06:19:54.691860 5449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:19:54.692281 5444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:19:54.692555 5444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:19:54.693733 5444 hybrid_clock.cc:648] HybridClock initialized: now 1790403594693725 us; error 41 us; skew 500 ppm
Sep 26 06:19:54 dist-test-slave-pd0t krb5kdc[5431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403594, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:19:54.702050 5444 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:19:54.702461 5444 webserver.cc:460] Webserver started at http://127.4.202.190:46287/ using document root <none> and password file <none>
I20260926 06:19:54.702725 5444 fs_manager.cc:359] Metadata directory not provided
I20260926 06:19:54.702805 5444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:19:54.702958 5444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:19:54.703956 5444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "d3f00b2a21024db9854f4b3c56d832c1"
format_stamp: "Formatted at 2026-09-26 06:19:54 on dist-test-slave-pd0t"
server_key: "fb875602c5263079fe5e180d8e1769b7"
server_key_iv: "9924c860d097ded31fded97d5f71f8d5"
server_key_version: "encryptionkey@0"
I20260926 06:19:54.704313 5444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "d3f00b2a21024db9854f4b3c56d832c1"
format_stamp: "Formatted at 2026-09-26 06:19:54 on dist-test-slave-pd0t"
server_key: "fb875602c5263079fe5e180d8e1769b7"
server_key_iv: "9924c860d097ded31fded97d5f71f8d5"
server_key_version: "encryptionkey@0"
I20260926 06:19:54.705588 5444 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:19:54.706370 5459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:54.706557 5444 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:19:54.706694 5444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "d3f00b2a21024db9854f4b3c56d832c1"
format_stamp: "Formatted at 2026-09-26 06:19:54 on dist-test-slave-pd0t"
server_key: "fb875602c5263079fe5e180d8e1769b7"
server_key_iv: "9924c860d097ded31fded97d5f71f8d5"
server_key_version: "encryptionkey@0"
I20260926 06:19:54.706765 5444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:19:54.731376 5444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:19:54.733211 5444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:19:54.733316 5444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:19:54.736901 5444 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:39553
I20260926 06:19:54.736963 5511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:39553 every 8 connection(s)
I20260926 06:19:54.737303 5444 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:19:54.738144 5512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:19:54.740252 5512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3f00b2a21024db9854f4b3c56d832c1: Bootstrap starting.
I20260926 06:19:54.740525 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 5444
I20260926 06:19:54.740615 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:19:54.740778 4906 external_mini_cluster.cc:1428] Setting key d1ad7c28ef0c1a53d4743227a43d439d
I20260926 06:19:54.740948 5512 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3f00b2a21024db9854f4b3c56d832c1: Neither blocks nor log segments found. Creating new log.
I20260926 06:19:54.741283 5512 log.cc:824] T 00000000000000000000000000000000 P d3f00b2a21024db9854f4b3c56d832c1: Log is configured to *not* fsync() on all Append() calls
I20260926 06:19:54.741978 5512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3f00b2a21024db9854f4b3c56d832c1: No bootstrap required, opened a new log
I20260926 06:19:54.744279 5512 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3f00b2a21024db9854f4b3c56d832c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3f00b2a21024db9854f4b3c56d832c1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 39553 } }
I20260926 06:19:54.744446 5512 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3f00b2a21024db9854f4b3c56d832c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:19:54.744474 5512 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3f00b2a21024db9854f4b3c56d832c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3f00b2a21024db9854f4b3c56d832c1, State: Initialized, Role: FOLLOWER
I20260926 06:19:54.744618 5512 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3f00b2a21024db9854f4b3c56d832c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3f00b2a21024db9854f4b3c56d832c1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 39553 } }
I20260926 06:19:54.744689 5512 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3f00b2a21024db9854f4b3c56d832c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:19:54.744711 5512 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3f00b2a21024db9854f4b3c56d832c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:19:54.744774 5512 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3f00b2a21024db9854f4b3c56d832c1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:19:54.745297 5512 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3f00b2a21024db9854f4b3c56d832c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3f00b2a21024db9854f4b3c56d832c1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 39553 } }
I20260926 06:19:54.745404 5512 leader_election.cc:302] T 00000000000000000000000000000000 P d3f00b2a21024db9854f4b3c56d832c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3f00b2a21024db9854f4b3c56d832c1; no voters:
I20260926 06:19:54.745535 5512 leader_election.cc:288] T 00000000000000000000000000000000 P d3f00b2a21024db9854f4b3c56d832c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:19:54.745599 5517 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3f00b2a21024db9854f4b3c56d832c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:19:54.745738 5517 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3f00b2a21024db9854f4b3c56d832c1 [term 1 LEADER]: Becoming Leader. State: Replica: d3f00b2a21024db9854f4b3c56d832c1, State: Running, Role: LEADER
I20260926 06:19:54.745754 5512 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3f00b2a21024db9854f4b3c56d832c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:19:54.745841 5517 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3f00b2a21024db9854f4b3c56d832c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3f00b2a21024db9854f4b3c56d832c1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 39553 } }
I20260926 06:19:54.746146 5517 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3f00b2a21024db9854f4b3c56d832c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3f00b2a21024db9854f4b3c56d832c1. Latest consensus state: current_term: 1 leader_uuid: "d3f00b2a21024db9854f4b3c56d832c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3f00b2a21024db9854f4b3c56d832c1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 39553 } } }
I20260926 06:19:54.746233 5517 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3f00b2a21024db9854f4b3c56d832c1 [sys.catalog]: This master's current role is: LEADER
I20260926 06:19:54.746157 5519 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3f00b2a21024db9854f4b3c56d832c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3f00b2a21024db9854f4b3c56d832c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3f00b2a21024db9854f4b3c56d832c1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 39553 } } }
I20260926 06:19:54.746337 5519 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3f00b2a21024db9854f4b3c56d832c1 [sys.catalog]: This master's current role is: LEADER
I20260926 06:19:54.746454 5521 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:19:54.746935 5521 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 26 06:19:54 dist-test-slave-pd0t krb5kdc[5431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:19:54.748522 5521 catalog_manager.cc:1347] Generated new cluster ID: f13164d0e9bc4e458961542d9a00552f
I20260926 06:19:54.748571 5521 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:19:54.752274 5515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:54.741258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33328 (local address 127.4.202.190:39553)
0926 06:19:54.741472 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:54.741482 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:54.741511 (+ 29us) server_negotiation.cc:407] Connection header received
0926 06:19:54.742046 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:54.742053 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:54.742128 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:54.742270 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:19:54.742854 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:54.744294 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:54.745186 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:54.745792 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:54.746212 (+ 420us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:54.748851 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:19:54.748854 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:19:54.748855 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:19:54.748864 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:19:54.750737 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:54.751128 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:54.751130 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:54.751131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:54.751224 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:54.751430 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:54.751431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:54.751431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:54.751685 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:19:54.751822 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:54.751949 (+ 127us) server_negotiation.cc:299] Negotiation successful
0926 06:19:54.752064 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":72,"threads_started":1}
I20260926 06:19:54.761631 5521 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:19:54.762318 5521 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:19:54.772267 5521 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3f00b2a21024db9854f4b3c56d832c1: Generated new TSK 0
I20260926 06:19:54.772421 5521 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.202.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
I20260926 06:19:54.797755 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.129:0
--local_ip_for_outbound_sockets=127.4.202.129
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:39553
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:35755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:19:54.879823 5538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:19:54.880023 5538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:19:54.880045 5538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:19:54.881209 5538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:19:54.881250 5538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:19:54.881278 5538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.129
I20260926 06:19:54.882735 5538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:35755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:39553
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.129
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:19:54.882965 5538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:19:54.883145 5538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:19:54.887863 5544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:19:54.887871 5546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:19:54.887969 5538 server_base.cc:1034] running on GCE node
W20260926 06:19:54.887876 5543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:19:54.888273 5538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:19:54.888661 5538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:19:54.889832 5538 hybrid_clock.cc:648] HybridClock initialized: now 1790403594889820 us; error 49 us; skew 500 ppm
Sep 26 06:19:54 dist-test-slave-pd0t krb5kdc[5431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403594, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:19:54.892321 5538 init.cc:377] Logged in from keytab as kudu/127.4.202.129@KRBTEST.COM (short username kudu)
I20260926 06:19:54.892787 5538 webserver.cc:460] Webserver started at http://127.4.202.129:44595/ using document root <none> and password file <none>
I20260926 06:19:54.893007 5538 fs_manager.cc:359] Metadata directory not provided
I20260926 06:19:54.893106 5538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:19:54.893260 5538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:19:54.894237 5538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-0/data/instance:
uuid: "a71827efd7dc4a3a9e0d11f1e6555aaf"
format_stamp: "Formatted at 2026-09-26 06:19:54 on dist-test-slave-pd0t"
server_key: "5b7dcd1c94a7b688aba9472b405a26e7"
server_key_iv: "9f11209a4de1ac7d525770e98e6aaff1"
server_key_version: "encryptionkey@0"
I20260926 06:19:54.894572 5538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance:
uuid: "a71827efd7dc4a3a9e0d11f1e6555aaf"
format_stamp: "Formatted at 2026-09-26 06:19:54 on dist-test-slave-pd0t"
server_key: "5b7dcd1c94a7b688aba9472b405a26e7"
server_key_iv: "9f11209a4de1ac7d525770e98e6aaff1"
server_key_version: "encryptionkey@0"
I20260926 06:19:54.895859 5538 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:19:54.896524 5553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:54.896721 5538 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:19:54.896811 5538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-0/wal
uuid: "a71827efd7dc4a3a9e0d11f1e6555aaf"
format_stamp: "Formatted at 2026-09-26 06:19:54 on dist-test-slave-pd0t"
server_key: "5b7dcd1c94a7b688aba9472b405a26e7"
server_key_iv: "9f11209a4de1ac7d525770e98e6aaff1"
server_key_version: "encryptionkey@0"
I20260926 06:19:54.896888 5538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:19:54.914948 5538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:19:54.916625 5538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:19:54.916792 5538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:19:54.916994 5538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:19:54.917282 5538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:19:54.917335 5538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:54.917375 5538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:19:54.917402 5538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:54.923354 5538 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.129:45405
I20260926 06:19:54.923411 5666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.129:45405 every 8 connection(s)
I20260926 06:19:54.923715 5538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
Sep 26 06:19:54 dist-test-slave-pd0t krb5kdc[5431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403594, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:19:54.931983 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 5538
I20260926 06:19:54.932078 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance
I20260926 06:19:54.932235 4906 external_mini_cluster.cc:1428] Setting key 7157e736be8d9ca281836d016a700ccd
I20260926 06:19:54.932593 5515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:54.924340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:43431 (local address 127.4.202.190:39553)
0926 06:19:54.924383 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:54.924385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:54.924685 (+ 300us) server_negotiation.cc:407] Connection header received
0926 06:19:54.925579 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:54.925580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:54.925592 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:54.925647 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:19:54.926278 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:54.926809 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:54.927771 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:54.927969 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:54.928043 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:54.930000 (+ 1957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:19:54.930002 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:19:54.930002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:19:54.930008 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:19:54.931194 (+ 1186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:54.931542 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:54.931543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:54.931543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:54.931589 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:54.931852 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:54.931853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:54.931854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:54.932008 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:19:54.932095 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:54.932399 (+ 304us) server_negotiation.cc:299] Negotiation successful
0926 06:19:54.932489 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:19:54.932816 5669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:54.924330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:39553 (local address 127.4.202.129:43431)
0926 06:19:54.924490 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:54.924546 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:54.925469 (+ 923us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:54.925713 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:54.925717 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:54.925952 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:19:54.926187 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:54.926191 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:54.926893 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:54.926894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:54.927676 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:54.927678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:54.928051 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:54.928052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:54.928186 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:54.928616 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:19:54.928623 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:19:54.929916 (+ 1293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:19:54.931264 (+ 1348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:54.931269 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:54.931271 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:54.931471 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:54.931644 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:54.931644 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:54.931645 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:54.931744 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:54.932096 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:19:54.932102 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:19:54.932237 (+ 135us) client_negotiation.cc:769] Sending connection context
0926 06:19:54.932364 (+ 127us) client_negotiation.cc:240] Negotiation successful
0926 06:19:54.932509 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":75,"threads_started":1}
I20260926 06:19:54.933228 5667 heartbeater.cc:347] Connected to a master server at 127.4.202.190:39553
I20260926 06:19:54.933318 5667 heartbeater.cc:464] Registering TS with master...
I20260926 06:19:54.933496 5667 heartbeater.cc:511] Master 127.4.202.190:39553 requested a full tablet report, sending...
I20260926 06:19:54.934002 5476 ts_manager.cc:194] Registered new tserver with Master: a71827efd7dc4a3a9e0d11f1e6555aaf (127.4.202.129:45405)
I20260926 06:19:54.934971 5476 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.129@KRBTEST.COM'} at 127.4.202.129:43431
WARNING: no policy specified for kudu/127.4.202.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
I20260926 06:19:54.956797 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.130:0
--local_ip_for_outbound_sockets=127.4.202.130
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:39553
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:35755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:19:55.034533 5672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:19:55.034798 5672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:19:55.034837 5672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:19:55.035980 5672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:19:55.036060 5672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:19:55.036113 5672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.130
I20260926 06:19:55.037509 5672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:35755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:39553
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.130
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:19:55.037738 5672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:19:55.037955 5672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 06:19:55.042917 5672 server_base.cc:1034] running on GCE node
W20260926 06:19:55.042951 5678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:19:55.042927 5680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:19:55.043030 5677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:19:55.043412 5672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:19:55.043610 5672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:19:55.044772 5672 hybrid_clock.cc:648] HybridClock initialized: now 1790403595044762 us; error 37 us; skew 500 ppm
Sep 26 06:19:55 dist-test-slave-pd0t krb5kdc[5431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403595, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:19:55.047331 5672 init.cc:377] Logged in from keytab as kudu/127.4.202.130@KRBTEST.COM (short username kudu)
I20260926 06:19:55.047849 5672 webserver.cc:460] Webserver started at http://127.4.202.130:35847/ using document root <none> and password file <none>
I20260926 06:19:55.048082 5672 fs_manager.cc:359] Metadata directory not provided
I20260926 06:19:55.048154 5672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:19:55.048357 5672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:19:55.049319 5672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-1/data/instance:
uuid: "15ac1cdaffca47e081e967b8cab61f30"
format_stamp: "Formatted at 2026-09-26 06:19:55 on dist-test-slave-pd0t"
server_key: "70e91a7fc3cc5b92a413651b05c905f3"
server_key_iv: "3696a209ed0986b22f22bb77735ae6e4"
server_key_version: "encryptionkey@0"
I20260926 06:19:55.049679 5672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance:
uuid: "15ac1cdaffca47e081e967b8cab61f30"
format_stamp: "Formatted at 2026-09-26 06:19:55 on dist-test-slave-pd0t"
server_key: "70e91a7fc3cc5b92a413651b05c905f3"
server_key_iv: "3696a209ed0986b22f22bb77735ae6e4"
server_key_version: "encryptionkey@0"
I20260926 06:19:55.051011 5672 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 06:19:55.051669 5687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:55.051815 5672 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:55.051900 5672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-1/wal
uuid: "15ac1cdaffca47e081e967b8cab61f30"
format_stamp: "Formatted at 2026-09-26 06:19:55 on dist-test-slave-pd0t"
server_key: "70e91a7fc3cc5b92a413651b05c905f3"
server_key_iv: "3696a209ed0986b22f22bb77735ae6e4"
server_key_version: "encryptionkey@0"
I20260926 06:19:55.051983 5672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:19:55.072059 5672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:19:55.073800 5672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:19:55.073946 5672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:19:55.074150 5672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:19:55.074445 5672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:19:55.074501 5672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:55.074538 5672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:19:55.074565 5672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:55.080246 5672 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.130:43277
I20260926 06:19:55.080279 5800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.130:43277 every 8 connection(s)
I20260926 06:19:55.080557 5672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
I20260926 06:19:55.080909 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 5672
I20260926 06:19:55.080996 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance
I20260926 06:19:55.081151 4906 external_mini_cluster.cc:1428] Setting key 5ac33055e9e671b88e394f312fe32fd9
Sep 26 06:19:55 dist-test-slave-pd0t krb5kdc[5431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403595, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.202.131@KRBTEST.COM; defaulting to no policy
I20260926 06:19:55.091029 5515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:55.081267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:37015 (local address 127.4.202.190:39553)
0926 06:19:55.081318 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:55.081321 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:55.081598 (+ 277us) server_negotiation.cc:407] Connection header received
0926 06:19:55.082599 (+ 1001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:55.082601 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:55.082614 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:55.082678 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:19:55.083320 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.083852 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.085418 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.085613 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.085671 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:55.087993 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:19:55.087994 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:19:55.087995 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:19:55.088002 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:19:55.089479 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:55.089974 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:55.089975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:55.089975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:55.090045 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:55.090306 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:55.090307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:55.090307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:55.090418 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:19:55.090490 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:55.090826 (+ 336us) server_negotiation.cc:299] Negotiation successful
0926 06:19:55.090912 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 06:19:55.091207 5803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:55.081261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:39553 (local address 127.4.202.130:37015)
0926 06:19:55.081417 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:55.081460 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:55.082463 (+ 1003us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:55.082748 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:55.082751 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:55.082973 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:19:55.083225 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:55.083229 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.083947 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.083948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:55.085328 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:55.085330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.085700 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.085701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:55.085837 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:55.086345 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:19:55.086356 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:19:55.087884 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:19:55.089581 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:55.089587 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:55.089589 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:55.089891 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:55.090110 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:55.090111 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:55.090111 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:55.090237 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:55.090495 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:19:55.090501 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:19:55.090675 (+ 174us) client_negotiation.cc:769] Sending connection context
0926 06:19:55.090805 (+ 130us) client_negotiation.cc:240] Negotiation successful
0926 06:19:55.090960 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":88,"threads_started":1}
I20260926 06:19:55.091650 5801 heartbeater.cc:347] Connected to a master server at 127.4.202.190:39553
I20260926 06:19:55.091747 5801 heartbeater.cc:464] Registering TS with master...
I20260926 06:19:55.091929 5801 heartbeater.cc:511] Master 127.4.202.190:39553 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.131@KRBTEST.COM" created.
I20260926 06:19:55.092335 5476 ts_manager.cc:194] Registered new tserver with Master: 15ac1cdaffca47e081e967b8cab61f30 (127.4.202.130:43277)
I20260926 06:19:55.092757 5476 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.130@KRBTEST.COM'} at 127.4.202.130:37015
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
I20260926 06:19:55.106114 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.131:0
--local_ip_for_outbound_sockets=127.4.202.131
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:39553
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:35755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:19:55.184373 5806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:19:55.184563 5806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:19:55.184594 5806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:19:55.185734 5806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:19:55.185773 5806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:19:55.185801 5806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.131
I20260926 06:19:55.187286 5806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:35755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:39553
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.131
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:19:55.187520 5806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:19:55.187758 5806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 06:19:55.192433 5806 server_base.cc:1034] running on GCE node
W20260926 06:19:55.192418 5814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:19:55.192385 5811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:19:55.192410 5812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:19:55.192845 5806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:19:55.193077 5806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:19:55.194239 5806 hybrid_clock.cc:648] HybridClock initialized: now 1790403595194216 us; error 40 us; skew 500 ppm
Sep 26 06:19:55 dist-test-slave-pd0t krb5kdc[5431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403595, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:19:55.196547 5806 init.cc:377] Logged in from keytab as kudu/127.4.202.131@KRBTEST.COM (short username kudu)
I20260926 06:19:55.196952 5806 webserver.cc:460] Webserver started at http://127.4.202.131:36167/ using document root <none> and password file <none>
I20260926 06:19:55.197173 5806 fs_manager.cc:359] Metadata directory not provided
I20260926 06:19:55.197239 5806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:19:55.197383 5806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:19:55.198339 5806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-2/data/instance:
uuid: "81aabd7502844e84b122ec6de5a25a83"
format_stamp: "Formatted at 2026-09-26 06:19:55 on dist-test-slave-pd0t"
server_key: "ccb12d12c36c8b3178f18ef9e2fd4687"
server_key_iv: "46decccb7bf408e559a2dae988a2ec67"
server_key_version: "encryptionkey@0"
I20260926 06:19:55.198737 5806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance:
uuid: "81aabd7502844e84b122ec6de5a25a83"
format_stamp: "Formatted at 2026-09-26 06:19:55 on dist-test-slave-pd0t"
server_key: "ccb12d12c36c8b3178f18ef9e2fd4687"
server_key_iv: "46decccb7bf408e559a2dae988a2ec67"
server_key_version: "encryptionkey@0"
I20260926 06:19:55.200049 5806 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:19:55.200733 5821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:55.200907 5806 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:19:55.200995 5806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-2/wal
uuid: "81aabd7502844e84b122ec6de5a25a83"
format_stamp: "Formatted at 2026-09-26 06:19:55 on dist-test-slave-pd0t"
server_key: "ccb12d12c36c8b3178f18ef9e2fd4687"
server_key_iv: "46decccb7bf408e559a2dae988a2ec67"
server_key_version: "encryptionkey@0"
I20260926 06:19:55.201076 5806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:19:55.219535 5806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:19:55.221259 5806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:19:55.221417 5806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:19:55.221626 5806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:19:55.221920 5806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:19:55.221972 5806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:55.222005 5806 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:19:55.222031 5806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:55.227514 5806 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.131:35931
I20260926 06:19:55.227556 5934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.131:35931 every 8 connection(s)
I20260926 06:19:55.227851 5806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
I20260926 06:19:55.230185 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 5806
I20260926 06:19:55.230297 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance
I20260926 06:19:55.230453 4906 external_mini_cluster.cc:1428] Setting key e69b0738e946a11b52dba4d3c8d76cad
Sep 26 06:19:55 dist-test-slave-pd0t krb5kdc[5431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403595, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:19:55.236979 5515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:55.228507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:43759 (local address 127.4.202.190:39553)
0926 06:19:55.228590 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:55.228593 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:55.228772 (+ 179us) server_negotiation.cc:407] Connection header received
0926 06:19:55.229743 (+ 971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:55.229744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:55.229756 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:55.229817 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:19:55.230414 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.231146 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.232159 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.232367 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.232467 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:55.234811 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:19:55.234812 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:19:55.234812 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:19:55.234818 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:19:55.235927 (+ 1109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:55.236270 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:55.236271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:55.236271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:55.236319 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:55.236519 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:55.236519 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:55.236519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:55.236608 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:19:55.236655 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:55.236866 (+ 211us) server_negotiation.cc:299] Negotiation successful
0926 06:19:55.236920 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 06:19:55.237289 5937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:55.228466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:39553 (local address 127.4.202.131:43759)
0926 06:19:55.228630 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:55.228677 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:55.229624 (+ 947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:55.229879 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:55.229882 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:55.230134 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:19:55.230341 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:55.230347 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.231227 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.231228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:55.232060 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:55.232062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.232477 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.232478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:55.232619 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:55.233198 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:19:55.233209 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:19:55.234731 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:19:55.235994 (+ 1263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:55.235999 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:55.236001 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:55.236209 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:55.236380 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:55.236381 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:55.236381 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:55.236468 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:55.236659 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:19:55.236662 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:19:55.236763 (+ 101us) client_negotiation.cc:769] Sending connection context
0926 06:19:55.236838 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 06:19:55.236975 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":84,"threads_started":1}
I20260926 06:19:55.237720 5935 heartbeater.cc:347] Connected to a master server at 127.4.202.190:39553
I20260926 06:19:55.237819 5935 heartbeater.cc:464] Registering TS with master...
I20260926 06:19:55.238024 5935 heartbeater.cc:511] Master 127.4.202.190:39553 requested a full tablet report, sending...
I20260926 06:19:55.238417 5476 ts_manager.cc:194] Registered new tserver with Master: 81aabd7502844e84b122ec6de5a25a83 (127.4.202.131:35931)
I20260926 06:19:55.238837 5476 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.131@KRBTEST.COM'} at 127.4.202.131:43759
I20260926 06:19:55.242199 4906 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:19:55.249567 5515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:55.243742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33344 (local address 127.4.202.190:39553)
0926 06:19:55.243805 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:55.243822 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:55.243931 (+ 109us) server_negotiation.cc:407] Connection header received
0926 06:19:55.244011 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:55.244012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:55.244021 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:55.244063 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:19:55.244580 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.245014 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.245824 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.246037 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.246104 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:55.247333 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:19:55.247334 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:19:55.247335 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:19:55.247341 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:19:55.248650 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:55.248930 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:55.248930 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:55.248930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:55.248976 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:55.249130 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:55.249130 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:55.249131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:55.249238 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:19:55.249310 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:55.249411 (+ 101us) server_negotiation.cc:299] Negotiation successful
0926 06:19:55.249460 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 06:19:55.251276 5476 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33344:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:19:55.252790 5476 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:19:55.258404 5943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:55.254747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:45405 (local address 127.0.0.1:40684)
0926 06:19:55.254886 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:55.254921 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:55.254979 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:55.255240 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:55.255242 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:55.255245 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:19:55.255313 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:55.255314 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.256306 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.256306 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:55.257246 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:55.257249 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.258138 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.258139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:55.258211 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:55.258223 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 06:19:55.258293 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 06:19:55.258338 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":52,"threads_started":1}
I20260926 06:19:55.258435 5946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:55.254847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40684 (local address 127.4.202.129:45405)
0926 06:19:55.255016 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:55.255021 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:55.255030 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:19:55.255065 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:55.255067 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:55.255080 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:55.255152 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:19:55.255406 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.256201 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.257357 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.258051 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.258121 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:55.258171 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:55.258319 (+ 148us) server_negotiation.cc:299] Negotiation successful
0926 06:19:55.258359 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":70,"threads_started":1}
I20260926 06:19:55.258812 5947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:55.254933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:43277 (local address 127.0.0.1:36538)
0926 06:19:55.255193 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:55.255208 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:55.255250 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:55.255420 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:55.255420 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:55.255422 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:19:55.255495 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:55.255497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.256796 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.256797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:55.257640 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:55.257642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.258571 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.258572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:55.258649 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:55.258653 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:19:55.258702 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 06:19:55.258749 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":41,"threads_started":1}
I20260926 06:19:55.258895 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:55.254748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:35931 (local address 127.0.0.1:40894)
0926 06:19:55.255021 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:55.255031 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:55.255079 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:55.255589 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:55.255590 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:55.255592 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:19:55.255631 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:55.255633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.256786 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.256787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:55.257836 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:55.257838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.258669 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.258670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:55.258734 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:55.258738 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:19:55.258792 (+ 54us) client_negotiation.cc:240] Negotiation successful
0926 06:19:55.258830 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":29,"thread_start_us":48,"threads_started":1}
I20260926 06:19:55.258898 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:55.254811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40894 (local address 127.4.202.131:35931)
0926 06:19:55.255436 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:55.255440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:55.255447 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:19:55.255475 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:55.255477 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:55.255489 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:55.255558 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:19:55.255730 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.256679 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.257921 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.258555 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.258667 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:55.258715 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:55.258804 (+ 89us) server_negotiation.cc:299] Negotiation successful
0926 06:19:55.258835 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":64,"threads_started":1}
I20260926 06:19:55.259163 5948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:55.255037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36538 (local address 127.4.202.130:43277)
0926 06:19:55.255198 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:55.255201 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:55.255247 (+ 46us) server_negotiation.cc:407] Connection header received
0926 06:19:55.255320 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:55.255323 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:55.255337 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:55.255410 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:19:55.255705 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.256679 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.257729 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.258416 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.258483 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:55.258540 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:55.259013 (+ 473us) server_negotiation.cc:299] Negotiation successful
0926 06:19:55.259075 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":69,"threads_started":1}
I20260926 06:19:55.259395 5869 tablet_service.cc:1467] Processing CreateTablet for tablet 0792d284592b4f0bab7dbda5e4916fef (DEFAULT_TABLE table=test-table [id=9b3fc62ccbc84f489e995f526a034f58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:19:55.259395 5601 tablet_service.cc:1467] Processing CreateTablet for tablet 0792d284592b4f0bab7dbda5e4916fef (DEFAULT_TABLE table=test-table [id=9b3fc62ccbc84f489e995f526a034f58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:19:55.259701 5601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0792d284592b4f0bab7dbda5e4916fef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:19:55.259734 5869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0792d284592b4f0bab7dbda5e4916fef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:19:55.259738 5735 tablet_service.cc:1467] Processing CreateTablet for tablet 0792d284592b4f0bab7dbda5e4916fef (DEFAULT_TABLE table=test-table [id=9b3fc62ccbc84f489e995f526a034f58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:19:55.260052 5735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0792d284592b4f0bab7dbda5e4916fef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:19:55.262267 5949 tablet_bootstrap.cc:492] T 0792d284592b4f0bab7dbda5e4916fef P 15ac1cdaffca47e081e967b8cab61f30: Bootstrap starting.
I20260926 06:19:55.263023 5949 tablet_bootstrap.cc:654] T 0792d284592b4f0bab7dbda5e4916fef P 15ac1cdaffca47e081e967b8cab61f30: Neither blocks nor log segments found. Creating new log.
I20260926 06:19:55.262991 5951 tablet_bootstrap.cc:492] T 0792d284592b4f0bab7dbda5e4916fef P 81aabd7502844e84b122ec6de5a25a83: Bootstrap starting.
I20260926 06:19:55.263470 5949 log.cc:824] T 0792d284592b4f0bab7dbda5e4916fef P 15ac1cdaffca47e081e967b8cab61f30: Log is configured to *not* fsync() on all Append() calls
I20260926 06:19:55.263504 5951 tablet_bootstrap.cc:654] T 0792d284592b4f0bab7dbda5e4916fef P 81aabd7502844e84b122ec6de5a25a83: Neither blocks nor log segments found. Creating new log.
I20260926 06:19:55.263994 5951 log.cc:824] T 0792d284592b4f0bab7dbda5e4916fef P 81aabd7502844e84b122ec6de5a25a83: Log is configured to *not* fsync() on all Append() calls
I20260926 06:19:55.264065 5949 tablet_bootstrap.cc:492] T 0792d284592b4f0bab7dbda5e4916fef P 15ac1cdaffca47e081e967b8cab61f30: No bootstrap required, opened a new log
I20260926 06:19:55.264147 5949 ts_tablet_manager.cc:1397] T 0792d284592b4f0bab7dbda5e4916fef P 15ac1cdaffca47e081e967b8cab61f30: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:19:55.264274 5950 tablet_bootstrap.cc:492] T 0792d284592b4f0bab7dbda5e4916fef P a71827efd7dc4a3a9e0d11f1e6555aaf: Bootstrap starting.
I20260926 06:19:55.264627 5951 tablet_bootstrap.cc:492] T 0792d284592b4f0bab7dbda5e4916fef P 81aabd7502844e84b122ec6de5a25a83: No bootstrap required, opened a new log
I20260926 06:19:55.264691 5951 ts_tablet_manager.cc:1397] T 0792d284592b4f0bab7dbda5e4916fef P 81aabd7502844e84b122ec6de5a25a83: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:19:55.265218 5950 tablet_bootstrap.cc:654] T 0792d284592b4f0bab7dbda5e4916fef P a71827efd7dc4a3a9e0d11f1e6555aaf: Neither blocks nor log segments found. Creating new log.
I20260926 06:19:55.265642 5950 log.cc:824] T 0792d284592b4f0bab7dbda5e4916fef P a71827efd7dc4a3a9e0d11f1e6555aaf: Log is configured to *not* fsync() on all Append() calls
I20260926 06:19:55.266006 5949 raft_consensus.cc:348] T 0792d284592b4f0bab7dbda5e4916fef P 15ac1cdaffca47e081e967b8cab61f30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a71827efd7dc4a3a9e0d11f1e6555aaf" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45405 } } peers { permanent_uuid: "81aabd7502844e84b122ec6de5a25a83" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35931 } } peers { permanent_uuid: "15ac1cdaffca47e081e967b8cab61f30" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43277 } }
I20260926 06:19:55.266175 5949 raft_consensus.cc:374] T 0792d284592b4f0bab7dbda5e4916fef P 15ac1cdaffca47e081e967b8cab61f30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:19:55.266211 5949 raft_consensus.cc:729] T 0792d284592b4f0bab7dbda5e4916fef P 15ac1cdaffca47e081e967b8cab61f30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15ac1cdaffca47e081e967b8cab61f30, State: Initialized, Role: FOLLOWER
I20260926 06:19:55.266335 5949 consensus_queue.cc:260] T 0792d284592b4f0bab7dbda5e4916fef P 15ac1cdaffca47e081e967b8cab61f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a71827efd7dc4a3a9e0d11f1e6555aaf" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45405 } } peers { permanent_uuid: "81aabd7502844e84b122ec6de5a25a83" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35931 } } peers { permanent_uuid: "15ac1cdaffca47e081e967b8cab61f30" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43277 } }
I20260926 06:19:55.266538 5950 tablet_bootstrap.cc:492] T 0792d284592b4f0bab7dbda5e4916fef P a71827efd7dc4a3a9e0d11f1e6555aaf: No bootstrap required, opened a new log
I20260926 06:19:55.266610 5949 ts_tablet_manager.cc:1428] T 0792d284592b4f0bab7dbda5e4916fef P 15ac1cdaffca47e081e967b8cab61f30: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 06:19:55.266494 5951 raft_consensus.cc:348] T 0792d284592b4f0bab7dbda5e4916fef P 81aabd7502844e84b122ec6de5a25a83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a71827efd7dc4a3a9e0d11f1e6555aaf" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45405 } } peers { permanent_uuid: "81aabd7502844e84b122ec6de5a25a83" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35931 } } peers { permanent_uuid: "15ac1cdaffca47e081e967b8cab61f30" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43277 } }
I20260926 06:19:55.266651 5950 ts_tablet_manager.cc:1397] T 0792d284592b4f0bab7dbda5e4916fef P a71827efd7dc4a3a9e0d11f1e6555aaf: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:19:55.266662 5951 raft_consensus.cc:374] T 0792d284592b4f0bab7dbda5e4916fef P 81aabd7502844e84b122ec6de5a25a83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:19:55.266700 5951 raft_consensus.cc:729] T 0792d284592b4f0bab7dbda5e4916fef P 81aabd7502844e84b122ec6de5a25a83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81aabd7502844e84b122ec6de5a25a83, State: Initialized, Role: FOLLOWER
I20260926 06:19:55.266718 5801 heartbeater.cc:503] Master 127.4.202.190:39553 was elected leader, sending a full tablet report...
I20260926 06:19:55.266808 5951 consensus_queue.cc:260] T 0792d284592b4f0bab7dbda5e4916fef P 81aabd7502844e84b122ec6de5a25a83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a71827efd7dc4a3a9e0d11f1e6555aaf" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45405 } } peers { permanent_uuid: "81aabd7502844e84b122ec6de5a25a83" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35931 } } peers { permanent_uuid: "15ac1cdaffca47e081e967b8cab61f30" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43277 } }
I20260926 06:19:55.267050 5951 ts_tablet_manager.cc:1428] T 0792d284592b4f0bab7dbda5e4916fef P 81aabd7502844e84b122ec6de5a25a83: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 06:19:55.267171 5935 heartbeater.cc:503] Master 127.4.202.190:39553 was elected leader, sending a full tablet report...
I20260926 06:19:55.268445 5950 raft_consensus.cc:348] T 0792d284592b4f0bab7dbda5e4916fef P a71827efd7dc4a3a9e0d11f1e6555aaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a71827efd7dc4a3a9e0d11f1e6555aaf" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45405 } } peers { permanent_uuid: "81aabd7502844e84b122ec6de5a25a83" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35931 } } peers { permanent_uuid: "15ac1cdaffca47e081e967b8cab61f30" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43277 } }
I20260926 06:19:55.268596 5950 raft_consensus.cc:374] T 0792d284592b4f0bab7dbda5e4916fef P a71827efd7dc4a3a9e0d11f1e6555aaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:19:55.268647 5950 raft_consensus.cc:729] T 0792d284592b4f0bab7dbda5e4916fef P a71827efd7dc4a3a9e0d11f1e6555aaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a71827efd7dc4a3a9e0d11f1e6555aaf, State: Initialized, Role: FOLLOWER
I20260926 06:19:55.268752 5950 consensus_queue.cc:260] T 0792d284592b4f0bab7dbda5e4916fef P a71827efd7dc4a3a9e0d11f1e6555aaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a71827efd7dc4a3a9e0d11f1e6555aaf" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45405 } } peers { permanent_uuid: "81aabd7502844e84b122ec6de5a25a83" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35931 } } peers { permanent_uuid: "15ac1cdaffca47e081e967b8cab61f30" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43277 } }
I20260926 06:19:55.269004 5950 ts_tablet_manager.cc:1428] T 0792d284592b4f0bab7dbda5e4916fef P a71827efd7dc4a3a9e0d11f1e6555aaf: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 06:19:55.269025 5667 heartbeater.cc:503] Master 127.4.202.190:39553 was elected leader, sending a full tablet report...
W20260926 06:19:55.331287 5802 tablet.cc:2369] T 0792d284592b4f0bab7dbda5e4916fef P 15ac1cdaffca47e081e967b8cab61f30: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:19:55.368319 5957 raft_consensus.cc:482] T 0792d284592b4f0bab7dbda5e4916fef P a71827efd7dc4a3a9e0d11f1e6555aaf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:19:55.368484 5957 raft_consensus.cc:504] T 0792d284592b4f0bab7dbda5e4916fef P a71827efd7dc4a3a9e0d11f1e6555aaf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a71827efd7dc4a3a9e0d11f1e6555aaf" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45405 } } peers { permanent_uuid: "81aabd7502844e84b122ec6de5a25a83" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35931 } } peers { permanent_uuid: "15ac1cdaffca47e081e967b8cab61f30" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43277 } }
I20260926 06:19:55.368960 5957 leader_election.cc:288] T 0792d284592b4f0bab7dbda5e4916fef P a71827efd7dc4a3a9e0d11f1e6555aaf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 81aabd7502844e84b122ec6de5a25a83 (127.4.202.131:35931), 15ac1cdaffca47e081e967b8cab61f30 (127.4.202.130:43277)
I20260926 06:19:55.371869 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:55.369161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:48681 (local address 127.4.202.131:35931)
0926 06:19:55.369243 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:55.369245 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:55.369254 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:19:55.369278 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:55.369278 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:55.369287 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:55.369344 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:19:55.369569 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.370099 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.371131 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.371501 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.371554 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:55.371561 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:55.371751 (+ 190us) server_negotiation.cc:299] Negotiation successful
0926 06:19:55.371797 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 06:19:55.371884 5948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:55.369207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:58843 (local address 127.4.202.130:43277)
0926 06:19:55.369252 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:55.369254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:55.369446 (+ 192us) server_negotiation.cc:407] Connection header received
0926 06:19:55.369490 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:55.369491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:55.369504 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:55.369556 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:19:55.369683 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.370117 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.371138 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.371513 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.371604 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:55.371611 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:55.371780 (+ 169us) server_negotiation.cc:299] Negotiation successful
0926 06:19:55.371820 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 06:19:55.371884 5669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:55.369108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:35931 (local address 127.4.202.129:48681)
0926 06:19:55.369157 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:55.369170 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:55.369209 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:55.369363 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:55.369365 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:55.369367 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:19:55.369475 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:55.369477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.370203 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.370204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:55.371035 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:55.371037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.371585 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.371586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:55.371665 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:55.371668 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:19:55.371721 (+ 53us) client_negotiation.cc:240] Negotiation successful
0926 06:19:55.371776 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260926 06:19:55.371908 5958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:55.369115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:43277 (local address 127.4.202.129:58843)
0926 06:19:55.369343 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:55.369365 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:55.369414 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:55.369569 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:55.369569 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:55.369571 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:19:55.369631 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:55.369632 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.370237 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.370238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:55.371034 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:55.371036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.371624 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.371629 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:55.371683 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:55.371687 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:19:55.371761 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 06:19:55.371798 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":11,"thread_start_us":114,"threads_started":1}
I20260926 06:19:55.372284 5889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0792d284592b4f0bab7dbda5e4916fef" candidate_uuid: "a71827efd7dc4a3a9e0d11f1e6555aaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81aabd7502844e84b122ec6de5a25a83" is_pre_election: true
I20260926 06:19:55.372310 5755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0792d284592b4f0bab7dbda5e4916fef" candidate_uuid: "a71827efd7dc4a3a9e0d11f1e6555aaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15ac1cdaffca47e081e967b8cab61f30" is_pre_election: true
I20260926 06:19:55.372433 5755 raft_consensus.cc:2454] T 0792d284592b4f0bab7dbda5e4916fef P 15ac1cdaffca47e081e967b8cab61f30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a71827efd7dc4a3a9e0d11f1e6555aaf in term 0.
I20260926 06:19:55.372433 5889 raft_consensus.cc:2454] T 0792d284592b4f0bab7dbda5e4916fef P 81aabd7502844e84b122ec6de5a25a83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a71827efd7dc4a3a9e0d11f1e6555aaf in term 0.
I20260926 06:19:55.372650 5555 leader_election.cc:302] T 0792d284592b4f0bab7dbda5e4916fef P a71827efd7dc4a3a9e0d11f1e6555aaf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15ac1cdaffca47e081e967b8cab61f30, a71827efd7dc4a3a9e0d11f1e6555aaf; no voters:
I20260926 06:19:55.372793 5957 raft_consensus.cc:2781] T 0792d284592b4f0bab7dbda5e4916fef P a71827efd7dc4a3a9e0d11f1e6555aaf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:19:55.372848 5957 raft_consensus.cc:482] T 0792d284592b4f0bab7dbda5e4916fef P a71827efd7dc4a3a9e0d11f1e6555aaf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:19:55.372869 5957 raft_consensus.cc:3037] T 0792d284592b4f0bab7dbda5e4916fef P a71827efd7dc4a3a9e0d11f1e6555aaf [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:19:55.373577 5957 raft_consensus.cc:504] T 0792d284592b4f0bab7dbda5e4916fef P a71827efd7dc4a3a9e0d11f1e6555aaf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a71827efd7dc4a3a9e0d11f1e6555aaf" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45405 } } peers { permanent_uuid: "81aabd7502844e84b122ec6de5a25a83" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35931 } } peers { permanent_uuid: "15ac1cdaffca47e081e967b8cab61f30" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43277 } }
I20260926 06:19:55.373715 5957 leader_election.cc:288] T 0792d284592b4f0bab7dbda5e4916fef P a71827efd7dc4a3a9e0d11f1e6555aaf [CANDIDATE]: Term 1 election: Requested vote from peers 81aabd7502844e84b122ec6de5a25a83 (127.4.202.131:35931), 15ac1cdaffca47e081e967b8cab61f30 (127.4.202.130:43277)
I20260926 06:19:55.373862 5889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0792d284592b4f0bab7dbda5e4916fef" candidate_uuid: "a71827efd7dc4a3a9e0d11f1e6555aaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81aabd7502844e84b122ec6de5a25a83"
I20260926 06:19:55.373870 5755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0792d284592b4f0bab7dbda5e4916fef" candidate_uuid: "a71827efd7dc4a3a9e0d11f1e6555aaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15ac1cdaffca47e081e967b8cab61f30"
I20260926 06:19:55.373942 5889 raft_consensus.cc:3037] T 0792d284592b4f0bab7dbda5e4916fef P 81aabd7502844e84b122ec6de5a25a83 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:19:55.373944 5755 raft_consensus.cc:3037] T 0792d284592b4f0bab7dbda5e4916fef P 15ac1cdaffca47e081e967b8cab61f30 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:19:55.374686 5889 raft_consensus.cc:2454] T 0792d284592b4f0bab7dbda5e4916fef P 81aabd7502844e84b122ec6de5a25a83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a71827efd7dc4a3a9e0d11f1e6555aaf in term 1.
I20260926 06:19:55.374784 5755 raft_consensus.cc:2454] T 0792d284592b4f0bab7dbda5e4916fef P 15ac1cdaffca47e081e967b8cab61f30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a71827efd7dc4a3a9e0d11f1e6555aaf in term 1.
I20260926 06:19:55.374903 5554 leader_election.cc:302] T 0792d284592b4f0bab7dbda5e4916fef P a71827efd7dc4a3a9e0d11f1e6555aaf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81aabd7502844e84b122ec6de5a25a83, a71827efd7dc4a3a9e0d11f1e6555aaf; no voters:
I20260926 06:19:55.375002 5957 raft_consensus.cc:2781] T 0792d284592b4f0bab7dbda5e4916fef P a71827efd7dc4a3a9e0d11f1e6555aaf [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:19:55.375149 5957 raft_consensus.cc:686] T 0792d284592b4f0bab7dbda5e4916fef P a71827efd7dc4a3a9e0d11f1e6555aaf [term 1 LEADER]: Becoming Leader. State: Replica: a71827efd7dc4a3a9e0d11f1e6555aaf, State: Running, Role: LEADER
I20260926 06:19:55.375257 5957 consensus_queue.cc:237] T 0792d284592b4f0bab7dbda5e4916fef P a71827efd7dc4a3a9e0d11f1e6555aaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a71827efd7dc4a3a9e0d11f1e6555aaf" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45405 } } peers { permanent_uuid: "81aabd7502844e84b122ec6de5a25a83" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35931 } } peers { permanent_uuid: "15ac1cdaffca47e081e967b8cab61f30" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43277 } }
I20260926 06:19:55.376084 5475 catalog_manager.cc:5495] T 0792d284592b4f0bab7dbda5e4916fef P a71827efd7dc4a3a9e0d11f1e6555aaf reported cstate change: term changed from 0 to 1, leader changed from <none> to a71827efd7dc4a3a9e0d11f1e6555aaf (127.4.202.129). New cstate: current_term: 1 leader_uuid: "a71827efd7dc4a3a9e0d11f1e6555aaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a71827efd7dc4a3a9e0d11f1e6555aaf" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81aabd7502844e84b122ec6de5a25a83" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15ac1cdaffca47e081e967b8cab61f30" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43277 } health_report { overall_health: UNKNOWN } } }
Sep 26 06:19:55 dist-test-slave-pd0t krb5kdc[5431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.129@KRBTEST.COM
I20260926 06:19:55.405579 5946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:55.398372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40688 (local address 127.4.202.129:45405)
0926 06:19:55.398469 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:55.398472 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:55.398664 (+ 192us) server_negotiation.cc:407] Connection header received
0926 06:19:55.398716 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:55.398717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:55.398727 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:55.398800 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:19:55.399401 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.399925 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.400762 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.400953 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.401033 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:55.403296 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:19:55.403302 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:19:55.403302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:19:55.403309 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:19:55.404674 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:55.404968 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:55.404970 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:55.404971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:55.405022 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:55.405185 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:55.405186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:55.405186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:55.405302 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:19:55.405412 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:55.405457 (+ 45us) server_negotiation.cc:299] Negotiation successful
0926 06:19:55.405516 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
Sep 26 06:19:55 dist-test-slave-pd0t krb5kdc[5431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403595, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 06:19:55 dist-test-slave-pd0t krb5kdc[5431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403595, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.202.129@KRBTEST.COM
W20260926 06:19:55.424326 5668 tablet.cc:2369] T 0792d284592b4f0bab7dbda5e4916fef P a71827efd7dc4a3a9e0d11f1e6555aaf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:19:55.426994 5946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:55.420290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40702 (local address 127.4.202.129:45405)
0926 06:19:55.420356 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:55.420358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:55.420444 (+ 86us) server_negotiation.cc:407] Connection header received
0926 06:19:55.420545 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:55.420546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:55.420555 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:55.420610 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:19:55.421161 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.421639 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.422351 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.422547 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.422665 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:55.424755 (+ 2090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:19:55.424757 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:19:55.424758 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:19:55.424764 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:19:55.425874 (+ 1110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:55.426207 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:55.426208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:55.426208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:55.426263 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:55.426467 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:55.426468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:55.426468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:55.426559 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:19:55.426708 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:55.426878 (+ 170us) server_negotiation.cc:299] Negotiation successful
0926 06:19:55.426939 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
Sep 26 06:19:55 dist-test-slave-pd0t krb5kdc[5431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 06:19:55 dist-test-slave-pd0t krb5kdc[5431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.129@KRBTEST.COM
I20260926 06:19:55.447279 5946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:55.440212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40710 (local address 127.4.202.129:45405)
0926 06:19:55.440266 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:55.440268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:55.440721 (+ 453us) server_negotiation.cc:407] Connection header received
0926 06:19:55.440757 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:55.440758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:55.440773 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:55.440814 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:19:55.441230 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.441649 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.442333 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:55.442500 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:55.442653 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:55.444704 (+ 2051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:19:55.444705 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:19:55.444705 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:19:55.444723 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:19:55.446148 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:55.446516 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:55.446517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:55.446517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:55.446571 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:55.446810 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:55.446811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:55.446811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:55.446905 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:19:55.447021 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:55.447119 (+ 98us) server_negotiation.cc:299] Negotiation successful
0926 06:19:55.447185 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260926 06:19:55.447777 5581 tablet_service.cc:2908] Rejecting scan request for tablet 0792d284592b4f0bab7dbda5e4916fef: Uninitialized: clean time has not yet been initialized
I20260926 06:19:55.448926 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 5538
I20260926 06:19:55.454347 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 5672
I20260926 06:19:55.458442 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 5806
I20260926 06:19:55.462832 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 5444
2026-09-26T06:19:55Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2000 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:19:55 dist-test-slave-pd0t krb5kdc[5973](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:19:55 dist-test-slave-pd0t krb5kdc[5973](info): set up 2 sockets
Sep 26 06:19:55 dist-test-slave-pd0t krb5kdc[5973](info): commencing operation
krb5kdc: starting...
W20260926 06:19:56.490051 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:19:56 dist-test-slave-pd0t krb5kdc[5973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:19:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:19:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.202.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
I20260926 06:19:56.604043 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:39273
--webserver_interface=127.4.202.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:43055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:39273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:19:56.680301 5986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:19:56.680480 5986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:19:56.680500 5986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:19:56.681562 5986 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:19:56.681602 5986 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 06:19:56.681622 5986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:19:56.681669 5986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:19:56.681685 5986 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:19:56.681710 5986 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:19:56.683221 5986 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:43055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:39273
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:39273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:19:56.683445 5986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:19:56.683670 5986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:19:56.688115 5994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:19:56.688184 5991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:19:56.688117 5992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:19:56.688249 5986 server_base.cc:1034] running on GCE node
I20260926 06:19:56.688592 5986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:19:56.688853 5986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:19:56.689985 5986 hybrid_clock.cc:648] HybridClock initialized: now 1790403596689965 us; error 34 us; skew 500 ppm
Sep 26 06:19:56 dist-test-slave-pd0t krb5kdc[5973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403596, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:19:56.692270 5986 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:19:56.692655 5986 webserver.cc:460] Webserver started at http://127.4.202.190:37761/ using document root <none> and password file <none>
I20260926 06:19:56.692863 5986 fs_manager.cc:359] Metadata directory not provided
I20260926 06:19:56.692929 5986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:19:56.693069 5986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:19:56.693997 5986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "6cc2d95950004c9197e47303a97b9674"
format_stamp: "Formatted at 2026-09-26 06:19:56 on dist-test-slave-pd0t"
server_key: "f3d233d068272fc68b395dadc3ff0020"
server_key_iv: "5054608d723589b6de69a337d3de1d28"
server_key_version: "encryptionkey@0"
I20260926 06:19:56.694337 5986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "6cc2d95950004c9197e47303a97b9674"
format_stamp: "Formatted at 2026-09-26 06:19:56 on dist-test-slave-pd0t"
server_key: "f3d233d068272fc68b395dadc3ff0020"
server_key_iv: "5054608d723589b6de69a337d3de1d28"
server_key_version: "encryptionkey@0"
I20260926 06:19:56.695643 5986 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:19:56.696327 6001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:56.696491 5986 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:19:56.696581 5986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "6cc2d95950004c9197e47303a97b9674"
format_stamp: "Formatted at 2026-09-26 06:19:56 on dist-test-slave-pd0t"
server_key: "f3d233d068272fc68b395dadc3ff0020"
server_key_iv: "5054608d723589b6de69a337d3de1d28"
server_key_version: "encryptionkey@0"
I20260926 06:19:56.696663 5986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:19:56.706272 5986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:19:56.707978 5986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:19:56.708132 5986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:19:56.712332 5986 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:39273
I20260926 06:19:56.712419 6063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:39273 every 8 connection(s)
I20260926 06:19:56.712694 5986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:19:56.713321 6064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:19:56.715386 6064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cc2d95950004c9197e47303a97b9674: Bootstrap starting.
I20260926 06:19:56.715919 6064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6cc2d95950004c9197e47303a97b9674: Neither blocks nor log segments found. Creating new log.
I20260926 06:19:56.716202 6064 log.cc:824] T 00000000000000000000000000000000 P 6cc2d95950004c9197e47303a97b9674: Log is configured to *not* fsync() on all Append() calls
I20260926 06:19:56.716821 6064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cc2d95950004c9197e47303a97b9674: No bootstrap required, opened a new log
I20260926 06:19:56.717936 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 5986
I20260926 06:19:56.718041 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:19:56.718228 4906 external_mini_cluster.cc:1428] Setting key d9f819fa420d05eca1137787e9d52a0a
I20260926 06:19:56.719098 6064 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6cc2d95950004c9197e47303a97b9674 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cc2d95950004c9197e47303a97b9674" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 39273 } }
I20260926 06:19:56.719252 6064 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6cc2d95950004c9197e47303a97b9674 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:19:56.719313 6064 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6cc2d95950004c9197e47303a97b9674 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cc2d95950004c9197e47303a97b9674, State: Initialized, Role: FOLLOWER
I20260926 06:19:56.719470 6064 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6cc2d95950004c9197e47303a97b9674 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cc2d95950004c9197e47303a97b9674" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 39273 } }
I20260926 06:19:56.719566 6064 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6cc2d95950004c9197e47303a97b9674 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:19:56.719609 6064 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6cc2d95950004c9197e47303a97b9674 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:19:56.719660 6064 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6cc2d95950004c9197e47303a97b9674 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:19:56.720183 6064 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6cc2d95950004c9197e47303a97b9674 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cc2d95950004c9197e47303a97b9674" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 39273 } }
I20260926 06:19:56.720306 6064 leader_election.cc:302] T 00000000000000000000000000000000 P 6cc2d95950004c9197e47303a97b9674 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6cc2d95950004c9197e47303a97b9674; no voters:
I20260926 06:19:56.720472 6064 leader_election.cc:288] T 00000000000000000000000000000000 P 6cc2d95950004c9197e47303a97b9674 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:19:56.720517 6069 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6cc2d95950004c9197e47303a97b9674 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:19:56.720660 6069 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6cc2d95950004c9197e47303a97b9674 [term 1 LEADER]: Becoming Leader. State: Replica: 6cc2d95950004c9197e47303a97b9674, State: Running, Role: LEADER
I20260926 06:19:56.720718 6064 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6cc2d95950004c9197e47303a97b9674 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:19:56.720785 6069 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6cc2d95950004c9197e47303a97b9674 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cc2d95950004c9197e47303a97b9674" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 39273 } }
I20260926 06:19:56.721123 6069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cc2d95950004c9197e47303a97b9674 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6cc2d95950004c9197e47303a97b9674. Latest consensus state: current_term: 1 leader_uuid: "6cc2d95950004c9197e47303a97b9674" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cc2d95950004c9197e47303a97b9674" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 39273 } } }
I20260926 06:19:56.721210 6069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cc2d95950004c9197e47303a97b9674 [sys.catalog]: This master's current role is: LEADER
I20260926 06:19:56.721349 6070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cc2d95950004c9197e47303a97b9674 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6cc2d95950004c9197e47303a97b9674" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cc2d95950004c9197e47303a97b9674" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 39273 } } }
I20260926 06:19:56.721419 6070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cc2d95950004c9197e47303a97b9674 [sys.catalog]: This master's current role is: LEADER
I20260926 06:19:56.721540 6077 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:19:56.722312 6077 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:19:56.723821 6077 catalog_manager.cc:1347] Generated new cluster ID: 18fbb2ce4f6548d286e7abef9ab7cd1d
I20260926 06:19:56.723889 6077 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:19:56 dist-test-slave-pd0t krb5kdc[5973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:19:56.729280 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:56.718723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53782 (local address 127.4.202.190:39273)
0926 06:19:56.718918 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:56.718924 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:56.718946 (+ 22us) server_negotiation.cc:407] Connection header received
0926 06:19:56.719065 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:56.719069 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:56.719107 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:56.719243 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:19:56.719976 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:56.721234 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:56.722333 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:56.722953 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:56.723090 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:56.725750 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:19:56.725754 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:19:56.725754 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:19:56.725765 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:19:56.727695 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:56.728115 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:56.728118 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:56.728119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:56.728231 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:56.728424 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:56.728425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:56.728425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:56.728737 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:19:56.728907 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:56.728970 (+ 63us) server_negotiation.cc:299] Negotiation successful
0926 06:19:56.729100 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":92,"threads_started":1}
I20260926 06:19:56.731866 6077 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:19:56.732579 6077 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:19:56.742549 6077 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6cc2d95950004c9197e47303a97b9674: Generated new TSK 0
I20260926 06:19:56.742727 6077 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.202.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
I20260926 06:19:56.765977 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.129:0
--local_ip_for_outbound_sockets=127.4.202.129
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:39273
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:43055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:19:56.843400 6090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:19:56.843622 6090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:19:56.843662 6090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:19:56.844818 6090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:19:56.844878 6090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:19:56.844911 6090 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 06:19:56.844949 6090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.129
I20260926 06:19:56.846352 6090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:43055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.202.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.202.190:39273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.129
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:19:56.846661 6090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:19:56.846890 6090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:19:56.852023 6098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:19:56.852018 6096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:19:56.852102 6090 server_base.cc:1034] running on GCE node
W20260926 06:19:56.852018 6095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:19:56.852487 6090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:19:56.852736 6090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:19:56.853875 6090 hybrid_clock.cc:648] HybridClock initialized: now 1790403596853867 us; error 27 us; skew 500 ppm
Sep 26 06:19:56 dist-test-slave-pd0t krb5kdc[5973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403596, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:19:56.856402 6090 init.cc:377] Logged in from keytab as kudu/127.4.202.129@KRBTEST.COM (short username kudu)
I20260926 06:19:56.856818 6090 webserver.cc:460] Webserver started at http://127.4.202.129:44133/ using document root <none> and password file <none>
I20260926 06:19:56.857035 6090 fs_manager.cc:359] Metadata directory not provided
I20260926 06:19:56.857079 6090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:19:56.857249 6090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:19:56.858237 6090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-0/data/instance:
uuid: "e034223f3b9c44c094aa186a6b218724"
format_stamp: "Formatted at 2026-09-26 06:19:56 on dist-test-slave-pd0t"
server_key: "4fa666c933265fa74e584ebc12c63cd5"
server_key_iv: "a61a14f9555c22b145624434b5442670"
server_key_version: "encryptionkey@0"
I20260926 06:19:56.858592 6090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance:
uuid: "e034223f3b9c44c094aa186a6b218724"
format_stamp: "Formatted at 2026-09-26 06:19:56 on dist-test-slave-pd0t"
server_key: "4fa666c933265fa74e584ebc12c63cd5"
server_key_iv: "a61a14f9555c22b145624434b5442670"
server_key_version: "encryptionkey@0"
I20260926 06:19:56.859846 6090 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:19:56.860527 6105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:56.860705 6090 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:19:56.860785 6090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-0/wal
uuid: "e034223f3b9c44c094aa186a6b218724"
format_stamp: "Formatted at 2026-09-26 06:19:56 on dist-test-slave-pd0t"
server_key: "4fa666c933265fa74e584ebc12c63cd5"
server_key_iv: "a61a14f9555c22b145624434b5442670"
server_key_version: "encryptionkey@0"
I20260926 06:19:56.860852 6090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:19:56.872092 6090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:19:56.874246 6090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:19:56.874346 6090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:19:56.874848 6090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:19:56.874886 6090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:56.874907 6090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:19:56.874949 6090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 06:19:56 dist-test-slave-pd0t krb5kdc[5973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403596, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:19:56.881443 6090 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.129:45245
I20260926 06:19:56.881511 6220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.129:45245 every 8 connection(s)
I20260926 06:19:56.881790 6090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
I20260926 06:19:56.884670 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:56.875258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:34819 (local address 127.4.202.190:39273)
0926 06:19:56.875303 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:56.875305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:56.875508 (+ 203us) server_negotiation.cc:407] Connection header received
0926 06:19:56.875977 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:56.875979 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:56.875993 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:56.876047 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:19:56.876875 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:56.877715 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:56.879266 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:56.879612 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:56.879692 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:56.881914 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:19:56.881916 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:19:56.881917 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:19:56.881926 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:19:56.883249 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:56.883709 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:56.883710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:56.883711 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:56.883768 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:56.884019 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:56.884020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:56.884020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:56.884163 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:19:56.884238 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:56.884496 (+ 258us) server_negotiation.cc:299] Negotiation successful
0926 06:19:56.884581 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 06:19:56.884868 6114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:56.875250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:39273 (local address 127.4.202.129:34819)
0926 06:19:56.875403 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:56.875434 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:56.875874 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:56.876123 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:56.876127 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:56.876474 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:19:56.876747 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:56.876752 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:56.877811 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:56.877812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:56.879157 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:56.879160 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:56.879709 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:56.879710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:56.879807 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:56.880310 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:19:56.880321 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:19:56.881816 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:19:56.883342 (+ 1526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:56.883349 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:56.883351 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:56.883623 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:56.883833 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:56.883834 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:56.883834 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:56.883950 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:56.884230 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:19:56.884236 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:19:56.884368 (+ 132us) client_negotiation.cc:769] Sending connection context
0926 06:19:56.884474 (+ 106us) client_negotiation.cc:240] Negotiation successful
0926 06:19:56.884622 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
I20260926 06:19:56.885347 6221 heartbeater.cc:347] Connected to a master server at 127.4.202.190:39273
I20260926 06:19:56.885426 6221 heartbeater.cc:464] Registering TS with master...
I20260926 06:19:56.885614 6221 heartbeater.cc:511] Master 127.4.202.190:39273 requested a full tablet report, sending...
I20260926 06:19:56.886106 6017 ts_manager.cc:194] Registered new tserver with Master: e034223f3b9c44c094aa186a6b218724 (127.4.202.129:45245)
I20260926 06:19:56.886857 6017 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.129@KRBTEST.COM'} at 127.4.202.129:34819
I20260926 06:19:56.890482 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 6090
I20260926 06:19:56.890587 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance
I20260926 06:19:56.890765 4906 external_mini_cluster.cc:1428] Setting key 658c4ce3190c758d6472649638ec16ff
I20260926 06:19:56.891896 6227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:56.886765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:39273 (local address 127.4.202.129:36521)
0926 06:19:56.886883 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:56.886898 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:56.886943 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:56.887150 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:56.887151 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:56.887358 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:19:56.887418 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:56.887420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:56.888067 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:56.888068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:56.888559 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:56.888560 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:56.888870 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:56.888871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:56.888922 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:56.889246 (+ 324us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:19:56.889250 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:19:56.889496 (+ 246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:19:56.890889 (+ 1393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:56.890890 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:56.890891 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:56.891099 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:56.891308 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:56.891309 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:56.891310 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:56.891366 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:56.891665 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:19:56.891666 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:19:56.891702 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 06:19:56.891756 (+ 54us) client_negotiation.cc:240] Negotiation successful
0926 06:19:56.891830 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":64,"threads_started":1}
I20260926 06:19:56.892012 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:56.886830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:36521 (local address 127.4.202.190:39273)
0926 06:19:56.887048 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:56.887050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:56.887058 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:19:56.887076 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:56.887077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:56.887086 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:56.887130 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:19:56.887501 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:56.887979 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:56.888640 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:56.888811 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:56.888859 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:56.889565 (+ 706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:19:56.889566 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:19:56.889567 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:19:56.889572 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:19:56.890796 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:56.891179 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:56.891180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:56.891181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:56.891233 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:56.891434 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:56.891435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:56.891435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:56.891586 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:19:56.891668 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:56.891792 (+ 124us) server_negotiation.cc:299] Negotiation successful
0926 06:19:56.891877 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198}
WARNING: no policy specified for kudu/127.4.202.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
I20260926 06:19:56.913681 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.130:0
--local_ip_for_outbound_sockets=127.4.202.130
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:39273
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:43055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:19:56.995199 6230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:19:56.995386 6230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:19:56.995410 6230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:19:56.996567 6230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:19:56.996613 6230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:19:56.996634 6230 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 06:19:56.996711 6230 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.130
I20260926 06:19:56.998132 6230 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:43055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.202.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.202.190:39273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.130
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:19:56.998363 6230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:19:56.998634 6230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:19:57.003422 6236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:19:57.003398 6235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:19:57.003383 6238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:19:57.003476 6230 server_base.cc:1034] running on GCE node
I20260926 06:19:57.003929 6230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:19:57.004103 6230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:19:57.005234 6230 hybrid_clock.cc:648] HybridClock initialized: now 1790403597005225 us; error 29 us; skew 500 ppm
Sep 26 06:19:57 dist-test-slave-pd0t krb5kdc[5973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403597, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:19:57.007668 6230 init.cc:377] Logged in from keytab as kudu/127.4.202.130@KRBTEST.COM (short username kudu)
I20260926 06:19:57.008008 6230 webserver.cc:460] Webserver started at http://127.4.202.130:43589/ using document root <none> and password file <none>
I20260926 06:19:57.008230 6230 fs_manager.cc:359] Metadata directory not provided
I20260926 06:19:57.008275 6230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:19:57.008395 6230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:19:57.009368 6230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-1/data/instance:
uuid: "c4981f0949ba4226b842a5a9e8e49145"
format_stamp: "Formatted at 2026-09-26 06:19:57 on dist-test-slave-pd0t"
server_key: "a3ff77c7bc3dbcd2cd04627f1c19585f"
server_key_iv: "d7838bfd90e9b3359625799ca9f7ad10"
server_key_version: "encryptionkey@0"
I20260926 06:19:57.009732 6230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance:
uuid: "c4981f0949ba4226b842a5a9e8e49145"
format_stamp: "Formatted at 2026-09-26 06:19:57 on dist-test-slave-pd0t"
server_key: "a3ff77c7bc3dbcd2cd04627f1c19585f"
server_key_iv: "d7838bfd90e9b3359625799ca9f7ad10"
server_key_version: "encryptionkey@0"
I20260926 06:19:57.011183 6230 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260926 06:19:57.011884 6245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:57.012058 6230 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:19:57.012132 6230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-1/wal
uuid: "c4981f0949ba4226b842a5a9e8e49145"
format_stamp: "Formatted at 2026-09-26 06:19:57 on dist-test-slave-pd0t"
server_key: "a3ff77c7bc3dbcd2cd04627f1c19585f"
server_key_iv: "d7838bfd90e9b3359625799ca9f7ad10"
server_key_version: "encryptionkey@0"
I20260926 06:19:57.012182 6230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:19:57.022732 6230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:19:57.024791 6230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:19:57.024890 6230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:19:57.025277 6230 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:19:57.025314 6230 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:57.025440 6230 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:19:57.025473 6230 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:57.031968 6230 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.130:36335
I20260926 06:19:57.031983 6360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.130:36335 every 8 connection(s)
Sep 26 06:19:57 dist-test-slave-pd0t krb5kdc[5973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403597, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:19:57.032385 6230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
I20260926 06:19:57.035079 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.025681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:37595 (local address 127.4.202.190:39273)
0926 06:19:57.025730 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:57.025733 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:57.025951 (+ 218us) server_negotiation.cc:407] Connection header received
0926 06:19:57.026370 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:57.026371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:57.026387 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:57.026464 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:19:57.027262 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.028090 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.029639 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.029978 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.030054 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.032816 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:19:57.032817 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:19:57.032818 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:19:57.032825 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:19:57.033962 (+ 1137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:57.034299 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:57.034300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:57.034300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:57.034351 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:57.034513 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:57.034514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:57.034514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:57.034689 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:19:57.034748 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:57.034953 (+ 205us) server_negotiation.cc:299] Negotiation successful
0926 06:19:57.035018 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:19:57.035197 6254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.025693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:39273 (local address 127.4.202.130:37595)
0926 06:19:57.025828 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:57.025862 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:57.026279 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:57.026507 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:57.026510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:57.026869 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:19:57.027161 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.027168 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.028220 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.028221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.029537 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.029541 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.030070 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.030071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.030167 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.030738 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:19:57.030747 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:19:57.032679 (+ 1932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:19:57.034027 (+ 1348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:57.034033 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:57.034034 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:57.034232 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:57.034391 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:57.034392 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:57.034392 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:57.034465 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:57.034741 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:19:57.034745 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:19:57.034842 (+ 97us) client_negotiation.cc:769] Sending connection context
0926 06:19:57.034924 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 06:19:57.035026 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":79,"threads_started":1}
I20260926 06:19:57.035689 6361 heartbeater.cc:347] Connected to a master server at 127.4.202.190:39273
I20260926 06:19:57.035784 6361 heartbeater.cc:464] Registering TS with master...
I20260926 06:19:57.035936 6361 heartbeater.cc:511] Master 127.4.202.190:39273 requested a full tablet report, sending...
I20260926 06:19:57.036320 6018 ts_manager.cc:194] Registered new tserver with Master: c4981f0949ba4226b842a5a9e8e49145 (127.4.202.130:36335)
I20260926 06:19:57.036728 6018 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.130@KRBTEST.COM'} at 127.4.202.130:37595
I20260926 06:19:57.038080 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 6230
I20260926 06:19:57.038179 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance
I20260926 06:19:57.038329 4906 external_mini_cluster.cc:1428] Setting key 89d55ded961796f8e72e485536337275
I20260926 06:19:57.042567 6367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.036607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:39273 (local address 127.4.202.130:44755)
0926 06:19:57.036714 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:57.036723 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:57.036771 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:57.036950 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:57.036951 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:57.037128 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:19:57.037183 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.037185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.037827 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.037827 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.038668 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.038670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.039094 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.039095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.039192 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.039518 (+ 326us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:19:57.039523 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:19:57.039894 (+ 371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:19:57.041601 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:57.041619 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:57.041619 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:57.041776 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:57.042000 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:57.042001 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:57.042001 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:57.042047 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:57.042355 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:19:57.042355 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:19:57.042388 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 06:19:57.042436 (+ 48us) client_negotiation.cc:240] Negotiation successful
0926 06:19:57.042503 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":65,"threads_started":1}
I20260926 06:19:57.042699 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.036670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:44755 (local address 127.4.202.190:39273)
0926 06:19:57.036851 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:57.036853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:57.036861 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:19:57.036879 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:57.036880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:57.036888 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:57.036937 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:19:57.037259 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.037751 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.038752 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.039014 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.039074 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.039970 (+ 896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:19:57.039971 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:19:57.039971 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:19:57.039977 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:19:57.041513 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:57.041877 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:57.041878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:57.041879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:57.041936 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:57.042105 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:57.042106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:57.042106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:57.042290 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:19:57.042361 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:57.042463 (+ 102us) server_negotiation.cc:299] Negotiation successful
0926 06:19:57.042542 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163}
WARNING: no policy specified for kudu/127.4.202.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
I20260926 06:19:57.061703 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.131:0
--local_ip_for_outbound_sockets=127.4.202.131
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:39273
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:43055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:19:57.141456 6370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:19:57.141647 6370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:19:57.141669 6370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:19:57.142815 6370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:19:57.142853 6370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:19:57.142874 6370 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 06:19:57.142891 6370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.131
I20260926 06:19:57.144173 6370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:43055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.202.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.202.190:39273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.131
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:19:57.144333 6370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:19:57.144485 6370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:19:57.149318 6376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:19:57.149402 6378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:19:57.149348 6375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:19:57.149608 6370 server_base.cc:1034] running on GCE node
I20260926 06:19:57.149796 6370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:19:57.149991 6370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:19:57.151160 6370 hybrid_clock.cc:648] HybridClock initialized: now 1790403597151150 us; error 43 us; skew 500 ppm
Sep 26 06:19:57 dist-test-slave-pd0t krb5kdc[5973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403597, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:19:57.153820 6370 init.cc:377] Logged in from keytab as kudu/127.4.202.131@KRBTEST.COM (short username kudu)
I20260926 06:19:57.154271 6370 webserver.cc:460] Webserver started at http://127.4.202.131:34431/ using document root <none> and password file <none>
I20260926 06:19:57.154452 6370 fs_manager.cc:359] Metadata directory not provided
I20260926 06:19:57.154493 6370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:19:57.154680 6370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:19:57.155766 6370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-2/data/instance:
uuid: "721fd41bf4df4ceb8d2663c741761920"
format_stamp: "Formatted at 2026-09-26 06:19:57 on dist-test-slave-pd0t"
server_key: "a5748b3509e38390f25d9d5eead240e9"
server_key_iv: "7f4e429816ceb8de6a715d6c180ed87f"
server_key_version: "encryptionkey@0"
I20260926 06:19:57.156162 6370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance:
uuid: "721fd41bf4df4ceb8d2663c741761920"
format_stamp: "Formatted at 2026-09-26 06:19:57 on dist-test-slave-pd0t"
server_key: "a5748b3509e38390f25d9d5eead240e9"
server_key_iv: "7f4e429816ceb8de6a715d6c180ed87f"
server_key_version: "encryptionkey@0"
I20260926 06:19:57.157539 6370 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:19:57.158308 6385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:57.158469 6370 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:19:57.158594 6370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-2/wal
uuid: "721fd41bf4df4ceb8d2663c741761920"
format_stamp: "Formatted at 2026-09-26 06:19:57 on dist-test-slave-pd0t"
server_key: "a5748b3509e38390f25d9d5eead240e9"
server_key_iv: "7f4e429816ceb8de6a715d6c180ed87f"
server_key_version: "encryptionkey@0"
I20260926 06:19:57.158695 6370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:19:57.170900 6370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:19:57.172652 6370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:19:57.172821 6370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:19:57.173344 6370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:19:57.173404 6370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:57.173447 6370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:19:57.173480 6370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 06:19:57 dist-test-slave-pd0t krb5kdc[5973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403597, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:19:57.182006 6370 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.131:46087
I20260926 06:19:57.182078 6500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.131:46087 every 8 connection(s)
I20260926 06:19:57.182394 6370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
I20260926 06:19:57.184418 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.174011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:52063 (local address 127.4.202.190:39273)
0926 06:19:57.174092 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:57.174094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:57.174342 (+ 248us) server_negotiation.cc:407] Connection header received
0926 06:19:57.175134 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:57.175136 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:57.175152 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:57.175234 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:19:57.175956 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.176564 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.178179 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.178460 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.178609 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.181150 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:19:57.181152 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:19:57.181152 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:19:57.181160 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:19:57.182952 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:57.183461 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:57.183463 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:57.183463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:57.183525 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:57.183803 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:57.183804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:57.183805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:57.183957 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:19:57.184030 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:57.184272 (+ 242us) server_negotiation.cc:299] Negotiation successful
0926 06:19:57.184334 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 06:19:57.184495 6394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.173994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:39273 (local address 127.4.202.131:52063)
0926 06:19:57.174196 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:57.174245 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:57.174999 (+ 754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:57.175320 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:57.175326 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:57.175616 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:19:57.175857 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.175862 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.176675 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.176676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.178069 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.178072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.178628 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.178629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.178818 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.179324 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:19:57.179333 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:19:57.181058 (+ 1725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:19:57.183055 (+ 1997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:57.183060 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:57.183062 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:57.183328 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:57.183607 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:57.183609 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:57.183609 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:57.183726 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:57.184027 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:19:57.184032 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:19:57.184143 (+ 111us) client_negotiation.cc:769] Sending connection context
0926 06:19:57.184235 (+ 92us) client_negotiation.cc:240] Negotiation successful
0926 06:19:57.184344 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":110,"threads_started":1}
I20260926 06:19:57.185127 6501 heartbeater.cc:347] Connected to a master server at 127.4.202.190:39273
I20260926 06:19:57.185225 6501 heartbeater.cc:464] Registering TS with master...
I20260926 06:19:57.185442 6501 heartbeater.cc:511] Master 127.4.202.190:39273 requested a full tablet report, sending...
I20260926 06:19:57.185686 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 6370
I20260926 06:19:57.185799 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance
I20260926 06:19:57.185995 4906 external_mini_cluster.cc:1428] Setting key 8f5ea11f23c9a9bad877b774c0f86ac3
I20260926 06:19:57.186343 6017 ts_manager.cc:194] Registered new tserver with Master: 721fd41bf4df4ceb8d2663c741761920 (127.4.202.131:46087)
I20260926 06:19:57.186843 6017 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.131@KRBTEST.COM'} at 127.4.202.131:52063
I20260926 06:19:57.187538 4906 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:19:57.192965 6507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.186318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:39273 (local address 127.4.202.131:51287)
0926 06:19:57.186483 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:57.186493 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:57.186538 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:57.187394 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:57.187395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:57.187675 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:19:57.187745 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.187747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.188687 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.188688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.189186 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.189188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.189618 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.189619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.189685 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.190148 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:19:57.190152 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:19:57.190406 (+ 254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:19:57.191947 (+ 1541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:57.191948 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:57.191949 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:57.192117 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:57.192332 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:57.192333 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:57.192333 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:57.192394 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:57.192639 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:19:57.192640 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:19:57.192683 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 06:19:57.192776 (+ 93us) client_negotiation.cc:240] Negotiation successful
0926 06:19:57.192871 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
I20260926 06:19:57.192960 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.186281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:51287 (local address 127.4.202.190:39273)
0926 06:19:57.187142 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:57.187144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:57.187153 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:19:57.187174 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:57.187175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:57.187186 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:57.187248 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:19:57.187821 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.188577 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.189306 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.189508 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.189587 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.190506 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:19:57.190508 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:19:57.190508 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:19:57.190514 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:19:57.191852 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:57.192206 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:57.192208 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:57.192208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:57.192259 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:57.192457 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:57.192458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:57.192459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:57.192573 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:19:57.192648 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:57.192799 (+ 151us) server_negotiation.cc:299] Negotiation successful
0926 06:19:57.192869 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":838}
Sep 26 06:19:57 dist-test-slave-pd0t krb5kdc[5973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403597, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 06:19:57 dist-test-slave-pd0t krb5kdc[5973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403597, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:19:57.223135 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.214694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53786 (local address 127.4.202.190:39273)
0926 06:19:57.214782 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:57.214785 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:57.214796 (+ 11us) server_negotiation.cc:407] Connection header received
0926 06:19:57.214824 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:57.214826 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:57.214837 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:57.214913 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:19:57.215438 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.216214 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.217459 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.217645 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.217696 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.220402 (+ 2706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:19:57.220404 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:19:57.220404 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:19:57.220412 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:19:57.221826 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:57.222242 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:57.222243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:57.222244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:57.222342 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:57.222552 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:57.222553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:57.222553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:57.222743 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:19:57.222865 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:57.222965 (+ 100us) server_negotiation.cc:299] Negotiation successful
0926 06:19:57.223038 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 06:19:57.224784 6017 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53786:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:19:57.226418 6017 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:19:57.232751 6518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.228580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58814 (local address 127.4.202.129:45245)
0926 06:19:57.228951 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:57.228956 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:57.228966 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:19:57.229006 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:57.229008 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:57.229024 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:57.229117 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:19:57.229284 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.230441 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.231773 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.232273 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.232373 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.232433 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:57.232618 (+ 185us) server_negotiation.cc:299] Negotiation successful
0926 06:19:57.232657 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":79,"threads_started":1}
I20260926 06:19:57.232744 6515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.228478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:45245 (local address 127.0.0.1:58814)
0926 06:19:57.228668 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:57.228706 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:57.228788 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:57.229109 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:57.229111 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:57.229113 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:19:57.229198 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.229200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.230526 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.230527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.231686 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.231688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.232375 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.232376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.232462 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.232475 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 06:19:57.232606 (+ 131us) client_negotiation.cc:240] Negotiation successful
0926 06:19:57.232658 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":54,"threads_started":1}
I20260926 06:19:57.232890 6520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.228694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52226 (local address 127.4.202.130:36335)
0926 06:19:57.228978 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:57.228982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:57.229010 (+ 28us) server_negotiation.cc:407] Connection header received
0926 06:19:57.229047 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:57.229049 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:57.229063 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:57.229145 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:19:57.229410 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.230484 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.231605 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.232340 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.232414 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.232466 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:57.232744 (+ 278us) server_negotiation.cc:299] Negotiation successful
0926 06:19:57.232782 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":132,"threads_started":1}
I20260926 06:19:57.232975 6516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.228490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:46087 (local address 127.0.0.1:50780)
0926 06:19:57.228883 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:57.228893 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:57.228938 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:57.229312 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:57.229313 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:57.229315 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:19:57.229360 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.229361 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.230256 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.230257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.231574 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.231576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.232503 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.232504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.232601 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.232604 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:19:57.232655 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 06:19:57.232702 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"mutex_wait_us":39,"thread_start_us":47,"threads_started":1}
I20260926 06:19:57.233098 6517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.228491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:36335 (local address 127.0.0.1:52226)
0926 06:19:57.228904 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:57.228915 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:57.228973 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:57.229209 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:57.229210 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:57.229212 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:19:57.229332 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.229334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.230565 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.230566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.231511 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.231513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.232504 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.232504 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.232604 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.232606 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:19:57.232655 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 06:19:57.232702 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":3,"thread_start_us":45,"threads_started":1}
I20260926 06:19:57.233523 6519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.228707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50780 (local address 127.4.202.131:46087)
0926 06:19:57.228892 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:57.228897 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:57.228997 (+ 100us) server_negotiation.cc:407] Connection header received
0926 06:19:57.229036 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:57.229039 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:57.229055 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:57.229120 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:19:57.229476 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.230169 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.231646 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.232344 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.232414 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.232466 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:57.232782 (+ 316us) server_negotiation.cc:299] Negotiation successful
0926 06:19:57.233434 (+ 652us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":113,"threads_started":1}
I20260926 06:19:57.233523 6155 tablet_service.cc:1467] Processing CreateTablet for tablet 4bc1be1862fb4b1c9c84b5fcea99f1de (DEFAULT_TABLE table=test-table [id=7dff838a8624411eb3175a76f1937f00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:19:57.233845 6155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bc1be1862fb4b1c9c84b5fcea99f1de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:19:57.233790 6295 tablet_service.cc:1467] Processing CreateTablet for tablet 4bc1be1862fb4b1c9c84b5fcea99f1de (DEFAULT_TABLE table=test-table [id=7dff838a8624411eb3175a76f1937f00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:19:57.234114 6295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bc1be1862fb4b1c9c84b5fcea99f1de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:19:57.234252 6429 tablet_service.cc:1467] Processing CreateTablet for tablet 4bc1be1862fb4b1c9c84b5fcea99f1de (DEFAULT_TABLE table=test-table [id=7dff838a8624411eb3175a76f1937f00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:19:57.234539 6429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bc1be1862fb4b1c9c84b5fcea99f1de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:19:57.236820 6521 tablet_bootstrap.cc:492] T 4bc1be1862fb4b1c9c84b5fcea99f1de P e034223f3b9c44c094aa186a6b218724: Bootstrap starting.
I20260926 06:19:57.237356 6521 tablet_bootstrap.cc:654] T 4bc1be1862fb4b1c9c84b5fcea99f1de P e034223f3b9c44c094aa186a6b218724: Neither blocks nor log segments found. Creating new log.
I20260926 06:19:57.237609 6521 log.cc:824] T 4bc1be1862fb4b1c9c84b5fcea99f1de P e034223f3b9c44c094aa186a6b218724: Log is configured to *not* fsync() on all Append() calls
I20260926 06:19:57.237711 6522 tablet_bootstrap.cc:492] T 4bc1be1862fb4b1c9c84b5fcea99f1de P 721fd41bf4df4ceb8d2663c741761920: Bootstrap starting.
I20260926 06:19:57.238498 6521 tablet_bootstrap.cc:492] T 4bc1be1862fb4b1c9c84b5fcea99f1de P e034223f3b9c44c094aa186a6b218724: No bootstrap required, opened a new log
I20260926 06:19:57.238615 6521 ts_tablet_manager.cc:1397] T 4bc1be1862fb4b1c9c84b5fcea99f1de P e034223f3b9c44c094aa186a6b218724: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:19:57.238664 6522 tablet_bootstrap.cc:654] T 4bc1be1862fb4b1c9c84b5fcea99f1de P 721fd41bf4df4ceb8d2663c741761920: Neither blocks nor log segments found. Creating new log.
I20260926 06:19:57.239050 6522 log.cc:824] T 4bc1be1862fb4b1c9c84b5fcea99f1de P 721fd41bf4df4ceb8d2663c741761920: Log is configured to *not* fsync() on all Append() calls
I20260926 06:19:57.239385 6523 tablet_bootstrap.cc:492] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145: Bootstrap starting.
I20260926 06:19:57.239745 6522 tablet_bootstrap.cc:492] T 4bc1be1862fb4b1c9c84b5fcea99f1de P 721fd41bf4df4ceb8d2663c741761920: No bootstrap required, opened a new log
I20260926 06:19:57.239827 6522 ts_tablet_manager.cc:1397] T 4bc1be1862fb4b1c9c84b5fcea99f1de P 721fd41bf4df4ceb8d2663c741761920: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260926 06:19:57.239946 6523 tablet_bootstrap.cc:654] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145: Neither blocks nor log segments found. Creating new log.
I20260926 06:19:57.240272 6523 log.cc:824] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145: Log is configured to *not* fsync() on all Append() calls
I20260926 06:19:57.240803 6521 raft_consensus.cc:348] T 4bc1be1862fb4b1c9c84b5fcea99f1de P e034223f3b9c44c094aa186a6b218724 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } }
I20260926 06:19:57.241055 6523 tablet_bootstrap.cc:492] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145: No bootstrap required, opened a new log
I20260926 06:19:57.241040 6521 raft_consensus.cc:374] T 4bc1be1862fb4b1c9c84b5fcea99f1de P e034223f3b9c44c094aa186a6b218724 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:19:57.241091 6521 raft_consensus.cc:729] T 4bc1be1862fb4b1c9c84b5fcea99f1de P e034223f3b9c44c094aa186a6b218724 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e034223f3b9c44c094aa186a6b218724, State: Initialized, Role: FOLLOWER
I20260926 06:19:57.241123 6523 ts_tablet_manager.cc:1397] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:19:57.241227 6521 consensus_queue.cc:260] T 4bc1be1862fb4b1c9c84b5fcea99f1de P e034223f3b9c44c094aa186a6b218724 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } }
I20260926 06:19:57.241490 6521 ts_tablet_manager.cc:1428] T 4bc1be1862fb4b1c9c84b5fcea99f1de P e034223f3b9c44c094aa186a6b218724: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 06:19:57.241528 6221 heartbeater.cc:503] Master 127.4.202.190:39273 was elected leader, sending a full tablet report...
I20260926 06:19:57.241839 6522 raft_consensus.cc:348] T 4bc1be1862fb4b1c9c84b5fcea99f1de P 721fd41bf4df4ceb8d2663c741761920 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } }
I20260926 06:19:57.241999 6522 raft_consensus.cc:374] T 4bc1be1862fb4b1c9c84b5fcea99f1de P 721fd41bf4df4ceb8d2663c741761920 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:19:57.242040 6522 raft_consensus.cc:729] T 4bc1be1862fb4b1c9c84b5fcea99f1de P 721fd41bf4df4ceb8d2663c741761920 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 721fd41bf4df4ceb8d2663c741761920, State: Initialized, Role: FOLLOWER
I20260926 06:19:57.242164 6522 consensus_queue.cc:260] T 4bc1be1862fb4b1c9c84b5fcea99f1de P 721fd41bf4df4ceb8d2663c741761920 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } }
I20260926 06:19:57.242451 6501 heartbeater.cc:503] Master 127.4.202.190:39273 was elected leader, sending a full tablet report...
I20260926 06:19:57.242436 6522 ts_tablet_manager.cc:1428] T 4bc1be1862fb4b1c9c84b5fcea99f1de P 721fd41bf4df4ceb8d2663c741761920: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 06:19:57.243465 6523 raft_consensus.cc:348] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } }
I20260926 06:19:57.243593 6523 raft_consensus.cc:374] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:19:57.243616 6523 raft_consensus.cc:729] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4981f0949ba4226b842a5a9e8e49145, State: Initialized, Role: FOLLOWER
I20260926 06:19:57.243734 6523 consensus_queue.cc:260] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } }
I20260926 06:19:57.243994 6523 ts_tablet_manager.cc:1428] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 06:19:57.244009 6361 heartbeater.cc:503] Master 127.4.202.190:39273 was elected leader, sending a full tablet report...
W20260926 06:19:57.283013 6362 tablet.cc:2369] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:19:57.300061 6529 raft_consensus.cc:482] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:19:57.300172 6529 raft_consensus.cc:504] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } }
I20260926 06:19:57.300539 6529 leader_election.cc:288] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e034223f3b9c44c094aa186a6b218724 (127.4.202.129:45245), 721fd41bf4df4ceb8d2663c741761920 (127.4.202.131:46087)
I20260926 06:19:57.304682 6519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.300795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:47575 (local address 127.4.202.131:46087)
0926 06:19:57.300885 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:57.300887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:57.301032 (+ 145us) server_negotiation.cc:407] Connection header received
0926 06:19:57.301091 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:57.301092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:57.301102 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:57.301164 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:19:57.301297 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.302031 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.303658 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.304251 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.304347 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.304355 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:57.304588 (+ 233us) server_negotiation.cc:299] Negotiation successful
0926 06:19:57.304633 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 06:19:57.304680 6530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.300710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:46087 (local address 127.4.202.130:47575)
0926 06:19:57.300933 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:57.300941 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:57.300994 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:57.301165 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:57.301165 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:57.301167 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:19:57.301218 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.301220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.302139 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.302140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.303554 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.303557 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.304346 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.304347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.304445 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.304451 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 06:19:57.304527 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 06:19:57.304570 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":94,"thread_start_us":90,"threads_started":1}
I20260926 06:19:57.305109 6254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.300703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:45245 (local address 127.4.202.130:42187)
0926 06:19:57.300770 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:57.300782 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:57.300825 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:57.301127 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:57.301128 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:57.301130 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:19:57.301218 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.301220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.302139 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.302140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.303555 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.303557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.304351 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.304351 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.304949 (+ 598us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.304953 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:19:57.305010 (+ 57us) client_negotiation.cc:240] Negotiation successful
0926 06:19:57.305046 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":108}
I20260926 06:19:57.305229 6518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.300794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:42187 (local address 127.4.202.129:45245)
0926 06:19:57.300866 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:57.300868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:57.300876 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:19:57.300903 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:57.300904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:57.300964 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:57.301090 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:19:57.301297 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.302056 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.303657 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.304251 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.304347 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.304355 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:57.305104 (+ 749us) server_negotiation.cc:299] Negotiation successful
0926 06:19:57.305157 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 06:19:57.305260 6455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bc1be1862fb4b1c9c84b5fcea99f1de" candidate_uuid: "c4981f0949ba4226b842a5a9e8e49145" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "721fd41bf4df4ceb8d2663c741761920" is_pre_election: true
I20260926 06:19:57.305423 6455 raft_consensus.cc:2454] T 4bc1be1862fb4b1c9c84b5fcea99f1de P 721fd41bf4df4ceb8d2663c741761920 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4981f0949ba4226b842a5a9e8e49145 in term 0.
I20260926 06:19:57.305476 6175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bc1be1862fb4b1c9c84b5fcea99f1de" candidate_uuid: "c4981f0949ba4226b842a5a9e8e49145" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e034223f3b9c44c094aa186a6b218724" is_pre_election: true
I20260926 06:19:57.305612 6175 raft_consensus.cc:2454] T 4bc1be1862fb4b1c9c84b5fcea99f1de P e034223f3b9c44c094aa186a6b218724 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4981f0949ba4226b842a5a9e8e49145 in term 0.
I20260926 06:19:57.305728 6249 leader_election.cc:302] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 721fd41bf4df4ceb8d2663c741761920, c4981f0949ba4226b842a5a9e8e49145; no voters:
I20260926 06:19:57.305872 6529 raft_consensus.cc:2781] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:19:57.305962 6529 raft_consensus.cc:482] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:19:57.306012 6529 raft_consensus.cc:3037] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:19:57.306746 6529 raft_consensus.cc:504] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } }
I20260926 06:19:57.306898 6529 leader_election.cc:288] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145 [CANDIDATE]: Term 1 election: Requested vote from peers e034223f3b9c44c094aa186a6b218724 (127.4.202.129:45245), 721fd41bf4df4ceb8d2663c741761920 (127.4.202.131:46087)
I20260926 06:19:57.307037 6175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bc1be1862fb4b1c9c84b5fcea99f1de" candidate_uuid: "c4981f0949ba4226b842a5a9e8e49145" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e034223f3b9c44c094aa186a6b218724"
I20260926 06:19:57.307117 6175 raft_consensus.cc:3037] T 4bc1be1862fb4b1c9c84b5fcea99f1de P e034223f3b9c44c094aa186a6b218724 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:19:57.307140 6455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bc1be1862fb4b1c9c84b5fcea99f1de" candidate_uuid: "c4981f0949ba4226b842a5a9e8e49145" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "721fd41bf4df4ceb8d2663c741761920"
I20260926 06:19:57.307252 6455 raft_consensus.cc:3037] T 4bc1be1862fb4b1c9c84b5fcea99f1de P 721fd41bf4df4ceb8d2663c741761920 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:19:57.307844 6175 raft_consensus.cc:2454] T 4bc1be1862fb4b1c9c84b5fcea99f1de P e034223f3b9c44c094aa186a6b218724 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4981f0949ba4226b842a5a9e8e49145 in term 1.
I20260926 06:19:57.307961 6455 raft_consensus.cc:2454] T 4bc1be1862fb4b1c9c84b5fcea99f1de P 721fd41bf4df4ceb8d2663c741761920 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4981f0949ba4226b842a5a9e8e49145 in term 1.
I20260926 06:19:57.308012 6247 leader_election.cc:302] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4981f0949ba4226b842a5a9e8e49145, e034223f3b9c44c094aa186a6b218724; no voters:
I20260926 06:19:57.308095 6529 raft_consensus.cc:2781] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:19:57.308240 6529 raft_consensus.cc:686] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145 [term 1 LEADER]: Becoming Leader. State: Replica: c4981f0949ba4226b842a5a9e8e49145, State: Running, Role: LEADER
I20260926 06:19:57.308331 6529 consensus_queue.cc:237] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } }
I20260926 06:19:57.309130 6018 catalog_manager.cc:5495] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4981f0949ba4226b842a5a9e8e49145 (127.4.202.130). New cstate: current_term: 1 leader_uuid: "c4981f0949ba4226b842a5a9e8e49145" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } health_report { overall_health: HEALTHY } } }
I20260926 06:19:57.328758 6520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.326501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52234 (local address 127.4.202.130:36335)
0926 06:19:57.326540 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:57.326542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:57.326551 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:19:57.326700 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:57.326700 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:57.326714 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:57.326760 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:19:57.326860 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.327292 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.328020 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.328201 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.328252 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.328425 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:19:57.328506 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:19:57.328560 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:57.328630 (+ 70us) server_negotiation.cc:299] Negotiation successful
0926 06:19:57.328648 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 06:19:57.330291 6175 raft_consensus.cc:1261] T 4bc1be1862fb4b1c9c84b5fcea99f1de P e034223f3b9c44c094aa186a6b218724 [term 1 FOLLOWER]: Refusing update from remote peer c4981f0949ba4226b842a5a9e8e49145: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:19:57.330302 6455 raft_consensus.cc:1261] T 4bc1be1862fb4b1c9c84b5fcea99f1de P 721fd41bf4df4ceb8d2663c741761920 [term 1 FOLLOWER]: Refusing update from remote peer c4981f0949ba4226b842a5a9e8e49145: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:19:57.330646 6531 consensus_queue.cc:1035] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:19:57.330802 6529 consensus_queue.cc:1035] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:19:57.336390 6017 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:19:57.336489 6017 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:19:57.337218 6017 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6cc2d95950004c9197e47303a97b9674: Sending DeleteTablet for 3 replicas of tablet 4bc1be1862fb4b1c9c84b5fcea99f1de
I20260926 06:19:57.337598 6429 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bc1be1862fb4b1c9c84b5fcea99f1de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:19:57 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:50780
I20260926 06:19:57.337553 6295 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bc1be1862fb4b1c9c84b5fcea99f1de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:19:57 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:52226
I20260926 06:19:57.337779 6155 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bc1be1862fb4b1c9c84b5fcea99f1de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:19:57 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:58814
I20260926 06:19:57.337836 6017 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53786:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:19:57.337971 6017 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:19:57.337985 6543 tablet_replica.cc:330] T 4bc1be1862fb4b1c9c84b5fcea99f1de P 721fd41bf4df4ceb8d2663c741761920: stopping tablet replica
I20260926 06:19:57.338083 6543 raft_consensus.cc:2229] T 4bc1be1862fb4b1c9c84b5fcea99f1de P 721fd41bf4df4ceb8d2663c741761920 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:19:57.338191 6543 raft_consensus.cc:2258] T 4bc1be1862fb4b1c9c84b5fcea99f1de P 721fd41bf4df4ceb8d2663c741761920 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:19:57.338192 6545 tablet_replica.cc:330] T 4bc1be1862fb4b1c9c84b5fcea99f1de P e034223f3b9c44c094aa186a6b218724: stopping tablet replica
I20260926 06:19:57.338320 6545 raft_consensus.cc:2229] T 4bc1be1862fb4b1c9c84b5fcea99f1de P e034223f3b9c44c094aa186a6b218724 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:19:57.338442 6545 raft_consensus.cc:2258] T 4bc1be1862fb4b1c9c84b5fcea99f1de P e034223f3b9c44c094aa186a6b218724 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:19:57.338900 6545 ts_tablet_manager.cc:1905] T 4bc1be1862fb4b1c9c84b5fcea99f1de P e034223f3b9c44c094aa186a6b218724: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:19:57.339260 6544 tablet_replica.cc:330] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145: stopping tablet replica
I20260926 06:19:57.339349 6544 raft_consensus.cc:2229] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145 [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:19:57.339430 6544 raft_consensus.cc:2258] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:19:57.339651 6295 tablet_service.cc:1467] Processing CreateTablet for tablet fe79d38873d04d00a5d86ef995204f6a (DEFAULT_TABLE table=test-table [id=64619e61e9c54b71a5cf877cb5e61ffc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:19:57.339769 6295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe79d38873d04d00a5d86ef995204f6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:19:57.339762 6543 ts_tablet_manager.cc:1905] T 4bc1be1862fb4b1c9c84b5fcea99f1de P 721fd41bf4df4ceb8d2663c741761920: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:19:57.341024 6523 tablet_bootstrap.cc:492] T fe79d38873d04d00a5d86ef995204f6a P c4981f0949ba4226b842a5a9e8e49145: Bootstrap starting.
I20260926 06:19:57.341218 6545 ts_tablet_manager.cc:1918] T 4bc1be1862fb4b1c9c84b5fcea99f1de P e034223f3b9c44c094aa186a6b218724: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:19:57.341279 6545 log.cc:1196] T 4bc1be1862fb4b1c9c84b5fcea99f1de P e034223f3b9c44c094aa186a6b218724: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-0/wal/wals/4bc1be1862fb4b1c9c84b5fcea99f1de
I20260926 06:19:57.341470 6523 tablet_bootstrap.cc:654] T fe79d38873d04d00a5d86ef995204f6a P c4981f0949ba4226b842a5a9e8e49145: Neither blocks nor log segments found. Creating new log.
I20260926 06:19:57.341674 6545 ts_tablet_manager.cc:1939] T 4bc1be1862fb4b1c9c84b5fcea99f1de P e034223f3b9c44c094aa186a6b218724: Deleting consensus metadata
I20260926 06:19:57.341741 6544 ts_tablet_manager.cc:1905] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:19:57.342121 6155 tablet_service.cc:1467] Processing CreateTablet for tablet fe79d38873d04d00a5d86ef995204f6a (DEFAULT_TABLE table=test-table [id=64619e61e9c54b71a5cf877cb5e61ffc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:19:57.342224 6155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe79d38873d04d00a5d86ef995204f6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:19:57.343019 6544 ts_tablet_manager.cc:1918] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:19:57.343071 6544 log.cc:1196] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-1/wal/wals/4bc1be1862fb4b1c9c84b5fcea99f1de
I20260926 06:19:57.343230 6544 ts_tablet_manager.cc:1939] T 4bc1be1862fb4b1c9c84b5fcea99f1de P c4981f0949ba4226b842a5a9e8e49145: Deleting consensus metadata
I20260926 06:19:57.343405 6521 tablet_bootstrap.cc:492] T fe79d38873d04d00a5d86ef995204f6a P e034223f3b9c44c094aa186a6b218724: Bootstrap starting.
I20260926 06:19:57.343427 6003 catalog_manager.cc:4841] TS e034223f3b9c44c094aa186a6b218724 (127.4.202.129:45245): tablet 4bc1be1862fb4b1c9c84b5fcea99f1de (table test-table [id=7dff838a8624411eb3175a76f1937f00]) successfully deleted
I20260926 06:19:57.343777 6521 tablet_bootstrap.cc:654] T fe79d38873d04d00a5d86ef995204f6a P e034223f3b9c44c094aa186a6b218724: Neither blocks nor log segments found. Creating new log.
I20260926 06:19:57.343775 6429 tablet_service.cc:1467] Processing CreateTablet for tablet fe79d38873d04d00a5d86ef995204f6a (DEFAULT_TABLE table=test-table [id=64619e61e9c54b71a5cf877cb5e61ffc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:19:57.343931 6004 catalog_manager.cc:4841] TS c4981f0949ba4226b842a5a9e8e49145 (127.4.202.130:36335): tablet 4bc1be1862fb4b1c9c84b5fcea99f1de (table test-table [id=7dff838a8624411eb3175a76f1937f00]) successfully deleted
I20260926 06:19:57.344201 6523 tablet_bootstrap.cc:492] T fe79d38873d04d00a5d86ef995204f6a P c4981f0949ba4226b842a5a9e8e49145: No bootstrap required, opened a new log
I20260926 06:19:57.344251 6523 ts_tablet_manager.cc:1397] T fe79d38873d04d00a5d86ef995204f6a P c4981f0949ba4226b842a5a9e8e49145: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260926 06:19:57.344303 6521 tablet_bootstrap.cc:492] T fe79d38873d04d00a5d86ef995204f6a P e034223f3b9c44c094aa186a6b218724: No bootstrap required, opened a new log
I20260926 06:19:57.344350 6521 ts_tablet_manager.cc:1397] T fe79d38873d04d00a5d86ef995204f6a P e034223f3b9c44c094aa186a6b218724: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 06:19:57.344383 6523 raft_consensus.cc:348] T fe79d38873d04d00a5d86ef995204f6a P c4981f0949ba4226b842a5a9e8e49145 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } }
I20260926 06:19:57.344450 6523 raft_consensus.cc:374] T fe79d38873d04d00a5d86ef995204f6a P c4981f0949ba4226b842a5a9e8e49145 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:19:57.344480 6523 raft_consensus.cc:729] T fe79d38873d04d00a5d86ef995204f6a P c4981f0949ba4226b842a5a9e8e49145 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4981f0949ba4226b842a5a9e8e49145, State: Initialized, Role: FOLLOWER
I20260926 06:19:57.344484 6521 raft_consensus.cc:348] T fe79d38873d04d00a5d86ef995204f6a P e034223f3b9c44c094aa186a6b218724 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } }
I20260926 06:19:57.344523 6523 consensus_queue.cc:260] T fe79d38873d04d00a5d86ef995204f6a P c4981f0949ba4226b842a5a9e8e49145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } }
I20260926 06:19:57.344551 6521 raft_consensus.cc:374] T fe79d38873d04d00a5d86ef995204f6a P e034223f3b9c44c094aa186a6b218724 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:19:57.344586 6521 raft_consensus.cc:729] T fe79d38873d04d00a5d86ef995204f6a P e034223f3b9c44c094aa186a6b218724 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e034223f3b9c44c094aa186a6b218724, State: Initialized, Role: FOLLOWER
I20260926 06:19:57.344601 6523 ts_tablet_manager.cc:1428] T fe79d38873d04d00a5d86ef995204f6a P c4981f0949ba4226b842a5a9e8e49145: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 06:19:57.344628 6521 consensus_queue.cc:260] T fe79d38873d04d00a5d86ef995204f6a P e034223f3b9c44c094aa186a6b218724 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } }
I20260926 06:19:57.344735 6521 ts_tablet_manager.cc:1428] T fe79d38873d04d00a5d86ef995204f6a P e034223f3b9c44c094aa186a6b218724: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 06:19:57.344839 6429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe79d38873d04d00a5d86ef995204f6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:19:57.346012 6522 tablet_bootstrap.cc:492] T fe79d38873d04d00a5d86ef995204f6a P 721fd41bf4df4ceb8d2663c741761920: Bootstrap starting.
I20260926 06:19:57.346202 6543 ts_tablet_manager.cc:1918] T 4bc1be1862fb4b1c9c84b5fcea99f1de P 721fd41bf4df4ceb8d2663c741761920: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:19:57.346276 6543 log.cc:1196] T 4bc1be1862fb4b1c9c84b5fcea99f1de P 721fd41bf4df4ceb8d2663c741761920: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-2/wal/wals/4bc1be1862fb4b1c9c84b5fcea99f1de
I20260926 06:19:57.346432 6522 tablet_bootstrap.cc:654] T fe79d38873d04d00a5d86ef995204f6a P 721fd41bf4df4ceb8d2663c741761920: Neither blocks nor log segments found. Creating new log.
I20260926 06:19:57.346531 6543 ts_tablet_manager.cc:1939] T 4bc1be1862fb4b1c9c84b5fcea99f1de P 721fd41bf4df4ceb8d2663c741761920: Deleting consensus metadata
I20260926 06:19:57.346942 6005 catalog_manager.cc:4841] TS 721fd41bf4df4ceb8d2663c741761920 (127.4.202.131:46087): tablet 4bc1be1862fb4b1c9c84b5fcea99f1de (table test-table [id=7dff838a8624411eb3175a76f1937f00]) successfully deleted
I20260926 06:19:57.347090 6522 tablet_bootstrap.cc:492] T fe79d38873d04d00a5d86ef995204f6a P 721fd41bf4df4ceb8d2663c741761920: No bootstrap required, opened a new log
I20260926 06:19:57.347158 6522 ts_tablet_manager.cc:1397] T fe79d38873d04d00a5d86ef995204f6a P 721fd41bf4df4ceb8d2663c741761920: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260926 06:19:57.347709 6522 raft_consensus.cc:348] T fe79d38873d04d00a5d86ef995204f6a P 721fd41bf4df4ceb8d2663c741761920 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } }
I20260926 06:19:57.347800 6522 raft_consensus.cc:374] T fe79d38873d04d00a5d86ef995204f6a P 721fd41bf4df4ceb8d2663c741761920 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:19:57.347833 6522 raft_consensus.cc:729] T fe79d38873d04d00a5d86ef995204f6a P 721fd41bf4df4ceb8d2663c741761920 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 721fd41bf4df4ceb8d2663c741761920, State: Initialized, Role: FOLLOWER
I20260926 06:19:57.347880 6522 consensus_queue.cc:260] T fe79d38873d04d00a5d86ef995204f6a P 721fd41bf4df4ceb8d2663c741761920 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } }
I20260926 06:19:57.347968 6522 ts_tablet_manager.cc:1428] T fe79d38873d04d00a5d86ef995204f6a P 721fd41bf4df4ceb8d2663c741761920: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
W20260926 06:19:57.382470 6222 tablet.cc:2369] T fe79d38873d04d00a5d86ef995204f6a P e034223f3b9c44c094aa186a6b218724: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 06:19:57.433012 6502 tablet.cc:2369] T fe79d38873d04d00a5d86ef995204f6a P 721fd41bf4df4ceb8d2663c741761920: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:19:57.492781 6527 raft_consensus.cc:482] T fe79d38873d04d00a5d86ef995204f6a P e034223f3b9c44c094aa186a6b218724 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:19:57.492869 6527 raft_consensus.cc:504] T fe79d38873d04d00a5d86ef995204f6a P e034223f3b9c44c094aa186a6b218724 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } }
I20260926 06:19:57.493149 6527 leader_election.cc:288] T fe79d38873d04d00a5d86ef995204f6a P e034223f3b9c44c094aa186a6b218724 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4981f0949ba4226b842a5a9e8e49145 (127.4.202.130:36335), 721fd41bf4df4ceb8d2663c741761920 (127.4.202.131:46087)
I20260926 06:19:57.497108 6520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.493406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:47305 (local address 127.4.202.130:36335)
0926 06:19:57.493459 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:57.493461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:57.493529 (+ 68us) server_negotiation.cc:407] Connection header received
0926 06:19:57.493614 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:57.493615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:57.493627 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:57.493677 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:19:57.493850 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.494644 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.496166 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.496743 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.496808 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.496815 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:57.496997 (+ 182us) server_negotiation.cc:299] Negotiation successful
0926 06:19:57.497035 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 06:19:57.497108 6519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.493440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:37823 (local address 127.4.202.131:46087)
0926 06:19:57.493480 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:57.493482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:57.493541 (+ 59us) server_negotiation.cc:407] Connection header received
0926 06:19:57.493599 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:57.493600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:57.493609 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:57.493672 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:19:57.493847 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.494611 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.496166 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.496700 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.496794 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.496804 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:57.496997 (+ 193us) server_negotiation.cc:299] Negotiation successful
0926 06:19:57.497033 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 06:19:57.497171 6549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.493330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:46087 (local address 127.4.202.129:37823)
0926 06:19:57.493464 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:57.493476 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:57.493540 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:57.493709 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:57.493710 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:57.493711 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:19:57.493769 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.493771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.494710 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.494712 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.496059 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.496062 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.496823 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.496824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.496910 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.496917 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 06:19:57.496976 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 06:19:57.497036 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":93,"thread_start_us":70,"threads_started":1}
I20260926 06:19:57.497174 6550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.493330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:36335 (local address 127.4.202.129:47305)
0926 06:19:57.493464 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:57.493476 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:57.493540 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:57.493685 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:57.493687 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:57.493689 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:19:57.493769 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.493773 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.494716 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.494717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.496058 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.496062 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.496821 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.496823 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.496910 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.496917 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 06:19:57.496976 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 06:19:57.497036 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":99,"thread_start_us":58,"threads_started":1}
I20260926 06:19:57.497566 6315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe79d38873d04d00a5d86ef995204f6a" candidate_uuid: "e034223f3b9c44c094aa186a6b218724" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4981f0949ba4226b842a5a9e8e49145" is_pre_election: true
I20260926 06:19:57.497601 6455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe79d38873d04d00a5d86ef995204f6a" candidate_uuid: "e034223f3b9c44c094aa186a6b218724" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "721fd41bf4df4ceb8d2663c741761920" is_pre_election: true
I20260926 06:19:57.497686 6315 raft_consensus.cc:2454] T fe79d38873d04d00a5d86ef995204f6a P c4981f0949ba4226b842a5a9e8e49145 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e034223f3b9c44c094aa186a6b218724 in term 0.
I20260926 06:19:57.497711 6455 raft_consensus.cc:2454] T fe79d38873d04d00a5d86ef995204f6a P 721fd41bf4df4ceb8d2663c741761920 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e034223f3b9c44c094aa186a6b218724 in term 0.
I20260926 06:19:57.497884 6108 leader_election.cc:302] T fe79d38873d04d00a5d86ef995204f6a P e034223f3b9c44c094aa186a6b218724 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4981f0949ba4226b842a5a9e8e49145, e034223f3b9c44c094aa186a6b218724; no voters:
I20260926 06:19:57.498011 6527 raft_consensus.cc:2781] T fe79d38873d04d00a5d86ef995204f6a P e034223f3b9c44c094aa186a6b218724 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:19:57.498056 6527 raft_consensus.cc:482] T fe79d38873d04d00a5d86ef995204f6a P e034223f3b9c44c094aa186a6b218724 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:19:57.498083 6527 raft_consensus.cc:3037] T fe79d38873d04d00a5d86ef995204f6a P e034223f3b9c44c094aa186a6b218724 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:19:57.498749 6527 raft_consensus.cc:504] T fe79d38873d04d00a5d86ef995204f6a P e034223f3b9c44c094aa186a6b218724 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } }
I20260926 06:19:57.498893 6527 leader_election.cc:288] T fe79d38873d04d00a5d86ef995204f6a P e034223f3b9c44c094aa186a6b218724 [CANDIDATE]: Term 1 election: Requested vote from peers c4981f0949ba4226b842a5a9e8e49145 (127.4.202.130:36335), 721fd41bf4df4ceb8d2663c741761920 (127.4.202.131:46087)
I20260926 06:19:57.499044 6315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe79d38873d04d00a5d86ef995204f6a" candidate_uuid: "e034223f3b9c44c094aa186a6b218724" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4981f0949ba4226b842a5a9e8e49145"
I20260926 06:19:57.499056 6455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe79d38873d04d00a5d86ef995204f6a" candidate_uuid: "e034223f3b9c44c094aa186a6b218724" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "721fd41bf4df4ceb8d2663c741761920"
I20260926 06:19:57.499122 6455 raft_consensus.cc:3037] T fe79d38873d04d00a5d86ef995204f6a P 721fd41bf4df4ceb8d2663c741761920 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:19:57.499122 6315 raft_consensus.cc:3037] T fe79d38873d04d00a5d86ef995204f6a P c4981f0949ba4226b842a5a9e8e49145 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:19:57.499907 6315 raft_consensus.cc:2454] T fe79d38873d04d00a5d86ef995204f6a P c4981f0949ba4226b842a5a9e8e49145 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e034223f3b9c44c094aa186a6b218724 in term 1.
I20260926 06:19:57.499913 6455 raft_consensus.cc:2454] T fe79d38873d04d00a5d86ef995204f6a P 721fd41bf4df4ceb8d2663c741761920 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e034223f3b9c44c094aa186a6b218724 in term 1.
I20260926 06:19:57.500125 6108 leader_election.cc:302] T fe79d38873d04d00a5d86ef995204f6a P e034223f3b9c44c094aa186a6b218724 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4981f0949ba4226b842a5a9e8e49145, e034223f3b9c44c094aa186a6b218724; no voters:
I20260926 06:19:57.500222 6527 raft_consensus.cc:2781] T fe79d38873d04d00a5d86ef995204f6a P e034223f3b9c44c094aa186a6b218724 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:19:57.500331 6527 raft_consensus.cc:686] T fe79d38873d04d00a5d86ef995204f6a P e034223f3b9c44c094aa186a6b218724 [term 1 LEADER]: Becoming Leader. State: Replica: e034223f3b9c44c094aa186a6b218724, State: Running, Role: LEADER
I20260926 06:19:57.500440 6527 consensus_queue.cc:237] T fe79d38873d04d00a5d86ef995204f6a P e034223f3b9c44c094aa186a6b218724 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } }
I20260926 06:19:57.501178 6016 catalog_manager.cc:5495] T fe79d38873d04d00a5d86ef995204f6a P e034223f3b9c44c094aa186a6b218724 reported cstate change: term changed from 0 to 1, leader changed from <none> to e034223f3b9c44c094aa186a6b218724 (127.4.202.129). New cstate: current_term: 1 leader_uuid: "e034223f3b9c44c094aa186a6b218724" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } health_report { overall_health: UNKNOWN } } }
Sep 26 06:19:57 dist-test-slave-pd0t krb5kdc[5973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403596, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:19:57.553766 6558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.547208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:39273 (local address 127.0.0.1:53790)
0926 06:19:57.547336 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:57.547345 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:57.547401 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:57.547555 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:57.547555 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:57.547842 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:19:57.547889 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.547890 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.548505 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.548505 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.549005 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.549006 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.549355 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.549355 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.549395 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.549872 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:19:57.549878 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:19:57.551249 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:19:57.552858 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:57.552864 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:57.552866 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:57.553040 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:57.553254 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:57.553254 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:57.553254 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:57.553306 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:57.553507 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:19:57.553513 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:19:57.553549 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 06:19:57.553598 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 06:19:57.553682 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":62,"threads_started":1}
I20260926 06:19:57.553812 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.547269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53790 (local address 127.4.202.190:39273)
0926 06:19:57.547440 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:57.547442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:57.547454 (+ 12us) server_negotiation.cc:407] Connection header received
0926 06:19:57.547479 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:57.547479 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:57.547487 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:57.547539 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:19:57.547954 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.548442 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.549094 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.549297 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.549347 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.551328 (+ 1981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:19:57.551329 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:19:57.551329 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:19:57.551335 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:19:57.552785 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:57.553133 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:57.553135 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:57.553135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:57.553189 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:57.553360 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:57.553361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:57.553361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:57.553456 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:19:57.553505 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:57.553615 (+ 110us) server_negotiation.cc:299] Negotiation successful
0926 06:19:57.553725 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":144}
I20260926 06:19:57.555790 6016 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:53790:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 06:19:57.556003 6016 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:19:57.557384 6295 tablet_service.cc:1467] Processing CreateTablet for tablet acee9a88a466458690cd35e720753fc8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0d40bedb38b4473fa82da6477576e219]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 06:19:57.557379 6429 tablet_service.cc:1467] Processing CreateTablet for tablet acee9a88a466458690cd35e720753fc8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0d40bedb38b4473fa82da6477576e219]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 06:19:57.557399 6155 tablet_service.cc:1467] Processing CreateTablet for tablet acee9a88a466458690cd35e720753fc8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0d40bedb38b4473fa82da6477576e219]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 06:19:57.557531 6295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acee9a88a466458690cd35e720753fc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:19:57.557531 6429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acee9a88a466458690cd35e720753fc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:19:57.557547 6155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acee9a88a466458690cd35e720753fc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:19:57.558782 6521 tablet_bootstrap.cc:492] T acee9a88a466458690cd35e720753fc8 P e034223f3b9c44c094aa186a6b218724: Bootstrap starting.
I20260926 06:19:57.559003 6523 tablet_bootstrap.cc:492] T acee9a88a466458690cd35e720753fc8 P c4981f0949ba4226b842a5a9e8e49145: Bootstrap starting.
I20260926 06:19:57.559207 6521 tablet_bootstrap.cc:654] T acee9a88a466458690cd35e720753fc8 P e034223f3b9c44c094aa186a6b218724: Neither blocks nor log segments found. Creating new log.
I20260926 06:19:57.559235 6522 tablet_bootstrap.cc:492] T acee9a88a466458690cd35e720753fc8 P 721fd41bf4df4ceb8d2663c741761920: Bootstrap starting.
I20260926 06:19:57.559350 6523 tablet_bootstrap.cc:654] T acee9a88a466458690cd35e720753fc8 P c4981f0949ba4226b842a5a9e8e49145: Neither blocks nor log segments found. Creating new log.
I20260926 06:19:57.559653 6522 tablet_bootstrap.cc:654] T acee9a88a466458690cd35e720753fc8 P 721fd41bf4df4ceb8d2663c741761920: Neither blocks nor log segments found. Creating new log.
I20260926 06:19:57.559877 6521 tablet_bootstrap.cc:492] T acee9a88a466458690cd35e720753fc8 P e034223f3b9c44c094aa186a6b218724: No bootstrap required, opened a new log
I20260926 06:19:57.559942 6521 ts_tablet_manager.cc:1397] T acee9a88a466458690cd35e720753fc8 P e034223f3b9c44c094aa186a6b218724: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 06:19:57.560065 6523 tablet_bootstrap.cc:492] T acee9a88a466458690cd35e720753fc8 P c4981f0949ba4226b842a5a9e8e49145: No bootstrap required, opened a new log
I20260926 06:19:57.560118 6523 ts_tablet_manager.cc:1397] T acee9a88a466458690cd35e720753fc8 P c4981f0949ba4226b842a5a9e8e49145: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 06:19:57.560091 6521 raft_consensus.cc:348] T acee9a88a466458690cd35e720753fc8 P e034223f3b9c44c094aa186a6b218724 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } }
I20260926 06:19:57.560158 6521 raft_consensus.cc:374] T acee9a88a466458690cd35e720753fc8 P e034223f3b9c44c094aa186a6b218724 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:19:57.560178 6521 raft_consensus.cc:729] T acee9a88a466458690cd35e720753fc8 P e034223f3b9c44c094aa186a6b218724 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e034223f3b9c44c094aa186a6b218724, State: Initialized, Role: FOLLOWER
I20260926 06:19:57.560225 6521 consensus_queue.cc:260] T acee9a88a466458690cd35e720753fc8 P e034223f3b9c44c094aa186a6b218724 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } }
I20260926 06:19:57.560211 6523 raft_consensus.cc:348] T acee9a88a466458690cd35e720753fc8 P c4981f0949ba4226b842a5a9e8e49145 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } }
I20260926 06:19:57.560247 6522 tablet_bootstrap.cc:492] T acee9a88a466458690cd35e720753fc8 P 721fd41bf4df4ceb8d2663c741761920: No bootstrap required, opened a new log
I20260926 06:19:57.560266 6523 raft_consensus.cc:374] T acee9a88a466458690cd35e720753fc8 P c4981f0949ba4226b842a5a9e8e49145 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:19:57.560287 6523 raft_consensus.cc:729] T acee9a88a466458690cd35e720753fc8 P c4981f0949ba4226b842a5a9e8e49145 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4981f0949ba4226b842a5a9e8e49145, State: Initialized, Role: FOLLOWER
I20260926 06:19:57.560289 6522 ts_tablet_manager.cc:1397] T acee9a88a466458690cd35e720753fc8 P 721fd41bf4df4ceb8d2663c741761920: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260926 06:19:57.560312 6521 ts_tablet_manager.cc:1428] T acee9a88a466458690cd35e720753fc8 P e034223f3b9c44c094aa186a6b218724: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:57.560324 6523 consensus_queue.cc:260] T acee9a88a466458690cd35e720753fc8 P c4981f0949ba4226b842a5a9e8e49145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } }
I20260926 06:19:57.560355 6527 tablet_replica.cc:439] T acee9a88a466458690cd35e720753fc8 P e034223f3b9c44c094aa186a6b218724: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } } }
I20260926 06:19:57.560415 6523 ts_tablet_manager.cc:1428] T acee9a88a466458690cd35e720753fc8 P c4981f0949ba4226b842a5a9e8e49145: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:57.560438 6527 tablet_replica.cc:442] T acee9a88a466458690cd35e720753fc8 P e034223f3b9c44c094aa186a6b218724: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:19:57.560420 6522 raft_consensus.cc:348] T acee9a88a466458690cd35e720753fc8 P 721fd41bf4df4ceb8d2663c741761920 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } }
I20260926 06:19:57.560433 6529 tablet_replica.cc:439] T acee9a88a466458690cd35e720753fc8 P c4981f0949ba4226b842a5a9e8e49145: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } } }
I20260926 06:19:57.560475 6522 raft_consensus.cc:374] T acee9a88a466458690cd35e720753fc8 P 721fd41bf4df4ceb8d2663c741761920 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:19:57.560499 6522 raft_consensus.cc:729] T acee9a88a466458690cd35e720753fc8 P 721fd41bf4df4ceb8d2663c741761920 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 721fd41bf4df4ceb8d2663c741761920, State: Initialized, Role: FOLLOWER
I20260926 06:19:57.560500 6529 tablet_replica.cc:442] T acee9a88a466458690cd35e720753fc8 P c4981f0949ba4226b842a5a9e8e49145: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:19:57.560541 6522 consensus_queue.cc:260] T acee9a88a466458690cd35e720753fc8 P 721fd41bf4df4ceb8d2663c741761920 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } }
I20260926 06:19:57.560640 6522 ts_tablet_manager.cc:1428] T acee9a88a466458690cd35e720753fc8 P 721fd41bf4df4ceb8d2663c741761920: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:57.560885 6528 tablet_replica.cc:439] T acee9a88a466458690cd35e720753fc8 P 721fd41bf4df4ceb8d2663c741761920: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } } }
I20260926 06:19:57.560984 6528 tablet_replica.cc:442] T acee9a88a466458690cd35e720753fc8 P 721fd41bf4df4ceb8d2663c741761920: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:19:57.851429 6528 raft_consensus.cc:482] T acee9a88a466458690cd35e720753fc8 P 721fd41bf4df4ceb8d2663c741761920 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:19:57.851531 6528 raft_consensus.cc:504] T acee9a88a466458690cd35e720753fc8 P 721fd41bf4df4ceb8d2663c741761920 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } }
I20260926 06:19:57.851919 6528 leader_election.cc:288] T acee9a88a466458690cd35e720753fc8 P 721fd41bf4df4ceb8d2663c741761920 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e034223f3b9c44c094aa186a6b218724 (127.4.202.129:45245), c4981f0949ba4226b842a5a9e8e49145 (127.4.202.130:36335)
I20260926 06:19:57.854784 6520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.852179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:53291 (local address 127.4.202.130:36335)
0926 06:19:57.852225 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:57.852227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:57.852284 (+ 57us) server_negotiation.cc:407] Connection header received
0926 06:19:57.852368 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:57.852369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:57.852376 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:57.852445 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:19:57.852635 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.853063 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.854025 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.854394 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.854457 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.854463 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:57.854661 (+ 198us) server_negotiation.cc:299] Negotiation successful
0926 06:19:57.854697 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 06:19:57.854796 6564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.852192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:55717 (local address 127.4.202.129:45245)
0926 06:19:57.852286 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:57.852288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:57.852315 (+ 27us) server_negotiation.cc:407] Connection header received
0926 06:19:57.852344 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:57.852345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:57.852356 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:57.852413 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:19:57.852580 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.853055 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.854026 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.854418 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.854502 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.854508 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:57.854670 (+ 162us) server_negotiation.cc:299] Negotiation successful
0926 06:19:57.854726 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74,"thread_start_us":49,"threads_started":1}
I20260926 06:19:57.854816 6563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.852107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:36335 (local address 127.4.202.131:53291)
0926 06:19:57.852220 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:57.852231 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:57.852295 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:57.852463 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:57.852464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:57.852467 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:19:57.852557 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.852559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.853174 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.853175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.853936 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.853938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.854479 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.854480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.854551 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.854554 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:19:57.854652 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 06:19:57.854696 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":17,"thread_start_us":45,"threads_started":1}
I20260926 06:19:57.854822 6562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.852080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:45245 (local address 127.4.202.131:55717)
0926 06:19:57.852211 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:57.852223 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:57.852281 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:57.852435 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:57.852436 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:57.852438 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:19:57.852496 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.852501 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.853130 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.853131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.853934 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.853936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.854513 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.854514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.854569 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.854571 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:19:57.854665 (+ 94us) client_negotiation.cc:240] Negotiation successful
0926 06:19:57.854720 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":27,"thread_start_us":57,"threads_started":1}
I20260926 06:19:57.855032 6315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acee9a88a466458690cd35e720753fc8" candidate_uuid: "721fd41bf4df4ceb8d2663c741761920" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4981f0949ba4226b842a5a9e8e49145" is_pre_election: true
I20260926 06:19:57.855127 6315 raft_consensus.cc:2454] T acee9a88a466458690cd35e720753fc8 P c4981f0949ba4226b842a5a9e8e49145 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 721fd41bf4df4ceb8d2663c741761920 in term 0.
I20260926 06:19:57.855110 6175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acee9a88a466458690cd35e720753fc8" candidate_uuid: "721fd41bf4df4ceb8d2663c741761920" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e034223f3b9c44c094aa186a6b218724" is_pre_election: true
I20260926 06:19:57.855201 6175 raft_consensus.cc:2454] T acee9a88a466458690cd35e720753fc8 P e034223f3b9c44c094aa186a6b218724 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 721fd41bf4df4ceb8d2663c741761920 in term 0.
I20260926 06:19:57.855291 6388 leader_election.cc:302] T acee9a88a466458690cd35e720753fc8 P 721fd41bf4df4ceb8d2663c741761920 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 721fd41bf4df4ceb8d2663c741761920, c4981f0949ba4226b842a5a9e8e49145; no voters:
I20260926 06:19:57.855427 6528 raft_consensus.cc:2781] T acee9a88a466458690cd35e720753fc8 P 721fd41bf4df4ceb8d2663c741761920 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:19:57.855480 6528 raft_consensus.cc:482] T acee9a88a466458690cd35e720753fc8 P 721fd41bf4df4ceb8d2663c741761920 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:19:57.855499 6528 raft_consensus.cc:3037] T acee9a88a466458690cd35e720753fc8 P 721fd41bf4df4ceb8d2663c741761920 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:19:57.856086 6528 raft_consensus.cc:504] T acee9a88a466458690cd35e720753fc8 P 721fd41bf4df4ceb8d2663c741761920 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } }
I20260926 06:19:57.856204 6528 leader_election.cc:288] T acee9a88a466458690cd35e720753fc8 P 721fd41bf4df4ceb8d2663c741761920 [CANDIDATE]: Term 1 election: Requested vote from peers e034223f3b9c44c094aa186a6b218724 (127.4.202.129:45245), c4981f0949ba4226b842a5a9e8e49145 (127.4.202.130:36335)
I20260926 06:19:57.856309 6175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acee9a88a466458690cd35e720753fc8" candidate_uuid: "721fd41bf4df4ceb8d2663c741761920" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e034223f3b9c44c094aa186a6b218724"
I20260926 06:19:57.856333 6315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acee9a88a466458690cd35e720753fc8" candidate_uuid: "721fd41bf4df4ceb8d2663c741761920" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4981f0949ba4226b842a5a9e8e49145"
I20260926 06:19:57.856381 6175 raft_consensus.cc:3037] T acee9a88a466458690cd35e720753fc8 P e034223f3b9c44c094aa186a6b218724 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:19:57.856392 6315 raft_consensus.cc:3037] T acee9a88a466458690cd35e720753fc8 P c4981f0949ba4226b842a5a9e8e49145 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:19:57.857106 6175 raft_consensus.cc:2454] T acee9a88a466458690cd35e720753fc8 P e034223f3b9c44c094aa186a6b218724 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 721fd41bf4df4ceb8d2663c741761920 in term 1.
I20260926 06:19:57.857149 6315 raft_consensus.cc:2454] T acee9a88a466458690cd35e720753fc8 P c4981f0949ba4226b842a5a9e8e49145 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 721fd41bf4df4ceb8d2663c741761920 in term 1.
I20260926 06:19:57.857275 6387 leader_election.cc:302] T acee9a88a466458690cd35e720753fc8 P 721fd41bf4df4ceb8d2663c741761920 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 721fd41bf4df4ceb8d2663c741761920, e034223f3b9c44c094aa186a6b218724; no voters:
I20260926 06:19:57.857378 6528 raft_consensus.cc:2781] T acee9a88a466458690cd35e720753fc8 P 721fd41bf4df4ceb8d2663c741761920 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:19:57.857483 6528 raft_consensus.cc:686] T acee9a88a466458690cd35e720753fc8 P 721fd41bf4df4ceb8d2663c741761920 [term 1 LEADER]: Becoming Leader. State: Replica: 721fd41bf4df4ceb8d2663c741761920, State: Running, Role: LEADER
I20260926 06:19:57.857542 6528 consensus_queue.cc:237] T acee9a88a466458690cd35e720753fc8 P 721fd41bf4df4ceb8d2663c741761920 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } }
I20260926 06:19:57.857915 6565 tablet_replica.cc:439] T acee9a88a466458690cd35e720753fc8 P 721fd41bf4df4ceb8d2663c741761920: TxnStatusTablet state changed. Reason: New leader 721fd41bf4df4ceb8d2663c741761920. Latest consensus state: current_term: 1 leader_uuid: "721fd41bf4df4ceb8d2663c741761920" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } } }
I20260926 06:19:57.858067 6565 tablet_replica.cc:442] T acee9a88a466458690cd35e720753fc8 P 721fd41bf4df4ceb8d2663c741761920: This TxnStatusTablet replica's current role is: LEADER
I20260926 06:19:57.858232 6567 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 06:19:57.858173 6016 catalog_manager.cc:5495] T acee9a88a466458690cd35e720753fc8 P 721fd41bf4df4ceb8d2663c741761920 reported cstate change: term changed from 0 to 1, leader changed from <none> to 721fd41bf4df4ceb8d2663c741761920 (127.4.202.131). New cstate: current_term: 1 leader_uuid: "721fd41bf4df4ceb8d2663c741761920" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } health_report { overall_health: HEALTHY } } }
I20260926 06:19:57.858340 6567 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 06:19:57.908188 6527 consensus_queue.cc:1035] T fe79d38873d04d00a5d86ef995204f6a P e034223f3b9c44c094aa186a6b218724 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:19:57.910385 6551 consensus_queue.cc:1035] T fe79d38873d04d00a5d86ef995204f6a P e034223f3b9c44c094aa186a6b218724 [LEADER]: Connected to new peer: Peer: permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:19:57.914403 6519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.911656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50782 (local address 127.4.202.131:46087)
0926 06:19:57.911694 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:57.911696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:57.911782 (+ 86us) server_negotiation.cc:407] Connection header received
0926 06:19:57.911909 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:57.911910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:57.911927 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:57.911981 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:19:57.912128 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.912582 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.913623 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.913794 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.913845 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.914061 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:19:57.914169 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:19:57.914221 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:57.914328 (+ 107us) server_negotiation.cc:299] Negotiation successful
0926 06:19:57.914348 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 06:19:57.914425 6558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.911629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:46087 (local address 127.0.0.1:50782)
0926 06:19:57.911737 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:57.911748 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:57.911802 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:57.912008 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:57.912009 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:57.912011 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 06:19:57.912060 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.912063 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.912690 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.912691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.913528 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.913531 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.913867 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.913868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.913932 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.913948 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 06:19:57.914235 (+ 287us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 06:19:57.914244 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 06:19:57.914303 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 06:19:57.914349 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260926 06:19:57.915467 6315 raft_consensus.cc:1261] T acee9a88a466458690cd35e720753fc8 P c4981f0949ba4226b842a5a9e8e49145 [term 1 FOLLOWER]: Refusing update from remote peer 721fd41bf4df4ceb8d2663c741761920: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:19:57.915529 6175 raft_consensus.cc:1261] T acee9a88a466458690cd35e720753fc8 P e034223f3b9c44c094aa186a6b218724 [term 1 FOLLOWER]: Refusing update from remote peer 721fd41bf4df4ceb8d2663c741761920: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:19:57.915714 6565 consensus_queue.cc:1035] T acee9a88a466458690cd35e720753fc8 P 721fd41bf4df4ceb8d2663c741761920 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:19:57.915804 6528 consensus_queue.cc:1035] T acee9a88a466458690cd35e720753fc8 P 721fd41bf4df4ceb8d2663c741761920 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:19:57.916306 6551 tablet_replica.cc:439] T acee9a88a466458690cd35e720753fc8 P e034223f3b9c44c094aa186a6b218724: TxnStatusTablet state changed. Reason: New leader 721fd41bf4df4ceb8d2663c741761920. Latest consensus state: current_term: 1 leader_uuid: "721fd41bf4df4ceb8d2663c741761920" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } } }
I20260926 06:19:57.916338 6529 tablet_replica.cc:439] T acee9a88a466458690cd35e720753fc8 P c4981f0949ba4226b842a5a9e8e49145: TxnStatusTablet state changed. Reason: New leader 721fd41bf4df4ceb8d2663c741761920. Latest consensus state: current_term: 1 leader_uuid: "721fd41bf4df4ceb8d2663c741761920" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } } }
I20260926 06:19:57.916393 6551 tablet_replica.cc:442] T acee9a88a466458690cd35e720753fc8 P e034223f3b9c44c094aa186a6b218724: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:19:57.916419 6529 tablet_replica.cc:442] T acee9a88a466458690cd35e720753fc8 P c4981f0949ba4226b842a5a9e8e49145: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:19:57.916931 6528 tablet_replica.cc:439] T acee9a88a466458690cd35e720753fc8 P 721fd41bf4df4ceb8d2663c741761920: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "721fd41bf4df4ceb8d2663c741761920" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } } }
I20260926 06:19:57.917011 6528 tablet_replica.cc:442] T acee9a88a466458690cd35e720753fc8 P 721fd41bf4df4ceb8d2663c741761920: This TxnStatusTablet replica's current role is: LEADER
I20260926 06:19:57.917085 6573 mvcc.cc:205] Tried to move back new op lower bound from 7333493137059909632 to 7333493136824922112. Current Snapshot: MvccSnapshot[applied={T|T < 7333493137059909632}]
I20260926 06:19:57.917613 6551 tablet_replica.cc:439] T acee9a88a466458690cd35e720753fc8 P e034223f3b9c44c094aa186a6b218724: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "721fd41bf4df4ceb8d2663c741761920" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } } }
I20260926 06:19:57.917689 6551 tablet_replica.cc:442] T acee9a88a466458690cd35e720753fc8 P e034223f3b9c44c094aa186a6b218724: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:19:57.919080 6565 tablet_replica.cc:439] T acee9a88a466458690cd35e720753fc8 P 721fd41bf4df4ceb8d2663c741761920: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "721fd41bf4df4ceb8d2663c741761920" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } } }
I20260926 06:19:57.919180 6565 tablet_replica.cc:442] T acee9a88a466458690cd35e720753fc8 P 721fd41bf4df4ceb8d2663c741761920: This TxnStatusTablet replica's current role is: LEADER
I20260926 06:19:57.919328 6571 mvcc.cc:205] Tried to move back new op lower bound from 7333493137059909632 to 7333493136824922112. Current Snapshot: MvccSnapshot[applied={T|T < 7333493137059909632}]
I20260926 06:19:57.919409 6529 tablet_replica.cc:439] T acee9a88a466458690cd35e720753fc8 P c4981f0949ba4226b842a5a9e8e49145: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "721fd41bf4df4ceb8d2663c741761920" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e034223f3b9c44c094aa186a6b218724" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45245 } } peers { permanent_uuid: "c4981f0949ba4226b842a5a9e8e49145" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36335 } } peers { permanent_uuid: "721fd41bf4df4ceb8d2663c741761920" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 46087 } } }
I20260926 06:19:57.919492 6529 tablet_replica.cc:442] T acee9a88a466458690cd35e720753fc8 P c4981f0949ba4226b842a5a9e8e49145: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:19:57.922678 6564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.919266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58820 (local address 127.4.202.129:45245)
0926 06:19:57.919427 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:57.919429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:57.919659 (+ 230us) server_negotiation.cc:407] Connection header received
0926 06:19:57.919774 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:57.919775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:57.919795 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:57.919867 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:19:57.920024 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.920742 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.921853 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.922107 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.922165 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.922335 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:19:57.922417 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:19:57.922465 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:57.922538 (+ 73us) server_negotiation.cc:299] Negotiation successful
0926 06:19:57.922555 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260926 06:19:57.927380 6519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.924896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:45605 (local address 127.4.202.131:46087)
0926 06:19:57.924938 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:57.924939 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:57.925057 (+ 118us) server_negotiation.cc:407] Connection header received
0926 06:19:57.925082 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:57.925083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:57.925091 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:57.925155 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:19:57.925230 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.925878 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.926612 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.926874 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.926973 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.927167 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:19:57.927198 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:19:57.927257 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:57.927304 (+ 47us) server_negotiation.cc:299] Negotiation successful
0926 06:19:57.927320 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 06:19:57.927392 6585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.924833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:46087 (local address 127.4.202.129:45605)
0926 06:19:57.924980 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:57.924990 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:57.925048 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:57.925156 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:57.925157 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:57.925158 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 06:19:57.925189 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.925190 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.925960 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.925961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.926496 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.926498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.927022 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.927023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.927068 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.927080 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 06:19:57.927264 (+ 184us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 06:19:57.927266 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:19:57.927299 (+ 33us) client_negotiation.cc:240] Negotiation successful
0926 06:19:57.927330 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":88,"threads_started":1}
I20260926 06:19:57.937873 6564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.934721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:45361 (local address 127.4.202.129:45245)
0926 06:19:57.934761 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:57.934763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:57.934918 (+ 155us) server_negotiation.cc:407] Connection header received
0926 06:19:57.935017 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:57.935018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:57.935030 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:57.935059 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:19:57.935283 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.936002 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.937084 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.937373 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.937445 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.937622 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:19:57.937655 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:19:57.937720 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:57.937787 (+ 67us) server_negotiation.cc:299] Negotiation successful
0926 06:19:57.937806 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 06:19:57.937915 6588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.934684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:45245 (local address 127.4.202.131:45361)
0926 06:19:57.934849 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:57.934861 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:57.934945 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:57.935118 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:57.935119 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:57.935120 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 06:19:57.935190 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.935191 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.936103 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.936104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.936995 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:57.936998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.937445 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.937446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:57.937517 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.937530 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 06:19:57.937719 (+ 189us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 06:19:57.937721 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:19:57.937782 (+ 61us) client_negotiation.cc:240] Negotiation successful
0926 06:19:57.937823 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
I20260926 06:19:57.951655 6016 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:19:57.951771 6016 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:19:57.952394 6016 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6cc2d95950004c9197e47303a97b9674: Sending DeleteTablet for 3 replicas of tablet fe79d38873d04d00a5d86ef995204f6a
I20260926 06:19:57.952702 6429 tablet_service.cc:1514] Processing DeleteTablet for tablet fe79d38873d04d00a5d86ef995204f6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:19:57 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:50780
I20260926 06:19:57.952817 6155 tablet_service.cc:1514] Processing DeleteTablet for tablet fe79d38873d04d00a5d86ef995204f6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:19:57 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:58814
I20260926 06:19:57.952904 6591 tablet_replica.cc:330] T fe79d38873d04d00a5d86ef995204f6a P 721fd41bf4df4ceb8d2663c741761920: stopping tablet replica
I20260926 06:19:57.952953 6591 raft_consensus.cc:2229] T fe79d38873d04d00a5d86ef995204f6a P 721fd41bf4df4ceb8d2663c741761920 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:19:57.953001 6591 raft_consensus.cc:2258] T fe79d38873d04d00a5d86ef995204f6a P 721fd41bf4df4ceb8d2663c741761920 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:19:57.953183 6295 tablet_service.cc:1514] Processing DeleteTablet for tablet fe79d38873d04d00a5d86ef995204f6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:19:57 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:52226
I20260926 06:19:57.953666 6591 ts_tablet_manager.cc:1905] T fe79d38873d04d00a5d86ef995204f6a P 721fd41bf4df4ceb8d2663c741761920: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:19:57.954053 6592 tablet_replica.cc:330] T fe79d38873d04d00a5d86ef995204f6a P e034223f3b9c44c094aa186a6b218724: stopping tablet replica
I20260926 06:19:57.954185 6593 tablet_replica.cc:330] T fe79d38873d04d00a5d86ef995204f6a P c4981f0949ba4226b842a5a9e8e49145: stopping tablet replica
I20260926 06:19:57.954531 6593 raft_consensus.cc:2229] T fe79d38873d04d00a5d86ef995204f6a P c4981f0949ba4226b842a5a9e8e49145 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:19:57.954677 6592 raft_consensus.cc:2229] T fe79d38873d04d00a5d86ef995204f6a P e034223f3b9c44c094aa186a6b218724 [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:19:57.954963 6592 raft_consensus.cc:2258] T fe79d38873d04d00a5d86ef995204f6a P e034223f3b9c44c094aa186a6b218724 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:19:57.955065 6593 raft_consensus.cc:2258] T fe79d38873d04d00a5d86ef995204f6a P c4981f0949ba4226b842a5a9e8e49145 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:19:57.955410 6591 ts_tablet_manager.cc:1918] T fe79d38873d04d00a5d86ef995204f6a P 721fd41bf4df4ceb8d2663c741761920: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 06:19:57.955415 6592 ts_tablet_manager.cc:1905] T fe79d38873d04d00a5d86ef995204f6a P e034223f3b9c44c094aa186a6b218724: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:19:57.955474 6591 log.cc:1196] T fe79d38873d04d00a5d86ef995204f6a P 721fd41bf4df4ceb8d2663c741761920: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-2/wal/wals/fe79d38873d04d00a5d86ef995204f6a
I20260926 06:19:57.955715 6591 ts_tablet_manager.cc:1939] T fe79d38873d04d00a5d86ef995204f6a P 721fd41bf4df4ceb8d2663c741761920: Deleting consensus metadata
I20260926 06:19:57.955998 6005 catalog_manager.cc:4841] TS 721fd41bf4df4ceb8d2663c741761920 (127.4.202.131:46087): tablet fe79d38873d04d00a5d86ef995204f6a (table test-table [id=64619e61e9c54b71a5cf877cb5e61ffc]) successfully deleted
I20260926 06:19:57.957326 6592 ts_tablet_manager.cc:1918] T fe79d38873d04d00a5d86ef995204f6a P e034223f3b9c44c094aa186a6b218724: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 06:19:57.957445 6592 log.cc:1196] T fe79d38873d04d00a5d86ef995204f6a P e034223f3b9c44c094aa186a6b218724: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-0/wal/wals/fe79d38873d04d00a5d86ef995204f6a
I20260926 06:19:57.957773 6592 ts_tablet_manager.cc:1939] T fe79d38873d04d00a5d86ef995204f6a P e034223f3b9c44c094aa186a6b218724: Deleting consensus metadata
I20260926 06:19:57.958158 6003 catalog_manager.cc:4841] TS e034223f3b9c44c094aa186a6b218724 (127.4.202.129:45245): tablet fe79d38873d04d00a5d86ef995204f6a (table test-table [id=64619e61e9c54b71a5cf877cb5e61ffc]) successfully deleted
I20260926 06:19:57.958786 6593 ts_tablet_manager.cc:1905] T fe79d38873d04d00a5d86ef995204f6a P c4981f0949ba4226b842a5a9e8e49145: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:19:57.960466 6593 ts_tablet_manager.cc:1918] T fe79d38873d04d00a5d86ef995204f6a P c4981f0949ba4226b842a5a9e8e49145: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 06:19:57.960529 6593 log.cc:1196] T fe79d38873d04d00a5d86ef995204f6a P c4981f0949ba4226b842a5a9e8e49145: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790403591640726-4906-0/minicluster-data/ts-1/wal/wals/fe79d38873d04d00a5d86ef995204f6a
I20260926 06:19:57.960844 6593 ts_tablet_manager.cc:1939] T fe79d38873d04d00a5d86ef995204f6a P c4981f0949ba4226b842a5a9e8e49145: Deleting consensus metadata
I20260926 06:19:57.961146 6004 catalog_manager.cc:4841] TS c4981f0949ba4226b842a5a9e8e49145 (127.4.202.130:36335): tablet fe79d38873d04d00a5d86ef995204f6a (table test-table [id=64619e61e9c54b71a5cf877cb5e61ffc]) successfully deleted
Sep 26 06:19:57 dist-test-slave-pd0t krb5kdc[5973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403597, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.202.129@KRBTEST.COM
I20260926 06:19:57.971273 6564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.954453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58834 (local address 127.4.202.129:45245)
0926 06:19:57.954490 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:57.954492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:57.961251 (+ 6759us) server_negotiation.cc:407] Connection header received
0926 06:19:57.961340 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:57.961342 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:57.961353 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:57.961415 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:19:57.962154 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.962924 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.963994 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.964368 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.964458 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.967919 (+ 3461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:19:57.967927 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:19:57.967928 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:19:57.967935 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:19:57.969889 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:57.970305 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:57.970308 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:57.970309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:57.970381 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:57.970686 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:57.970687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:57.970688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:57.970850 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:19:57.971009 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:57.971082 (+ 73us) server_negotiation.cc:299] Negotiation successful
0926 06:19:57.971163 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260926 06:19:57.971515 6215 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58834
I20260926 06:19:57.980046 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:57.972413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53798 (local address 127.4.202.190:39273)
0926 06:19:57.972454 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:57.972456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:57.972670 (+ 214us) server_negotiation.cc:407] Connection header received
0926 06:19:57.972728 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:57.972729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:57.972741 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:57.972807 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:19:57.973407 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.974099 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.975114 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:57.975372 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:57.975447 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:57.976778 (+ 1331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:19:57.976779 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:19:57.976779 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:19:57.976787 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:19:57.978833 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:57.979213 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:57.979214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:57.979215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:57.979275 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:57.979485 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:57.979486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:57.979486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:57.979654 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:19:57.979756 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:57.979890 (+ 134us) server_negotiation.cc:299] Negotiation successful
0926 06:19:57.979962 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260926 06:19:57.980551 6016 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53798
I20260926 06:19:57.981432 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 6090
I20260926 06:19:57.987843 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 6230
I20260926 06:19:57.994071 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 6370
I20260926 06:19:57.999410 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 5986
2026-09-26T06:19:57Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2536 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:19:58 dist-test-slave-pd0t krb5kdc[6600](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:19:58 dist-test-slave-pd0t krb5kdc[6600](info): set up 2 sockets
Sep 26 06:19:58 dist-test-slave-pd0t krb5kdc[6600](info): commencing operation
krb5kdc: starting...
W20260926 06:19:59.034189 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:19:59 dist-test-slave-pd0t krb5kdc[6600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:19:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:19:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.202.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
I20260926 06:19:59.147708 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:38709
--webserver_interface=127.4.202.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:37801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:38709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:19:59.226507 6613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:19:59.226781 6613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:19:59.226820 6613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:19:59.228006 6613 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:19:59.228087 6613 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 06:19:59.228113 6613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:19:59.228147 6613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:19:59.228169 6613 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:19:59.228192 6613 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:19:59.229734 6613 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:37801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:38709
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:38709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:19:59.229988 6613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:19:59.230273 6613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:19:59.235133 6618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:19:59.235181 6619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:19:59.235131 6621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:19:59.235464 6613 server_base.cc:1034] running on GCE node
I20260926 06:19:59.235709 6613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:19:59.236033 6613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:19:59.237219 6613 hybrid_clock.cc:648] HybridClock initialized: now 1790403599237194 us; error 39 us; skew 500 ppm
Sep 26 06:19:59 dist-test-slave-pd0t krb5kdc[6600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403599, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:19:59.239816 6613 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:19:59.240188 6613 webserver.cc:460] Webserver started at http://127.4.202.190:43013/ using document root <none> and password file <none>
I20260926 06:19:59.240376 6613 fs_manager.cc:359] Metadata directory not provided
I20260926 06:19:59.240415 6613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:19:59.240499 6613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:19:59.241523 6613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "4707268ef11e42139ecf487c90adb27a"
format_stamp: "Formatted at 2026-09-26 06:19:59 on dist-test-slave-pd0t"
server_key: "1bca00d787a992a5ed48f0ea3cb5d5e6"
server_key_iv: "f984ea9a56b6d213a5a3765db537ede0"
server_key_version: "encryptionkey@0"
I20260926 06:19:59.241822 6613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "4707268ef11e42139ecf487c90adb27a"
format_stamp: "Formatted at 2026-09-26 06:19:59 on dist-test-slave-pd0t"
server_key: "1bca00d787a992a5ed48f0ea3cb5d5e6"
server_key_iv: "f984ea9a56b6d213a5a3765db537ede0"
server_key_version: "encryptionkey@0"
I20260926 06:19:59.243135 6613 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:19:59.243813 6628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:59.244048 6613 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:19:59.244151 6613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "4707268ef11e42139ecf487c90adb27a"
format_stamp: "Formatted at 2026-09-26 06:19:59 on dist-test-slave-pd0t"
server_key: "1bca00d787a992a5ed48f0ea3cb5d5e6"
server_key_iv: "f984ea9a56b6d213a5a3765db537ede0"
server_key_version: "encryptionkey@0"
I20260926 06:19:59.244225 6613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:19:59.261039 6613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:19:59.263417 6613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:19:59.263578 6613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:19:59.268016 6613 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:38709
I20260926 06:19:59.268080 6690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:38709 every 8 connection(s)
I20260926 06:19:59.268415 6613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:19:59.269061 6691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:19:59.271237 6691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4707268ef11e42139ecf487c90adb27a: Bootstrap starting.
I20260926 06:19:59.271800 6691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4707268ef11e42139ecf487c90adb27a: Neither blocks nor log segments found. Creating new log.
I20260926 06:19:59.271905 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 6613
I20260926 06:19:59.271991 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:19:59.272138 6691 log.cc:824] T 00000000000000000000000000000000 P 4707268ef11e42139ecf487c90adb27a: Log is configured to *not* fsync() on all Append() calls
I20260926 06:19:59.272156 4906 external_mini_cluster.cc:1428] Setting key 31e02afdad83b88fc762dac0169fffcc
I20260926 06:19:59.272919 6691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4707268ef11e42139ecf487c90adb27a: No bootstrap required, opened a new log
I20260926 06:19:59.275074 6691 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4707268ef11e42139ecf487c90adb27a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4707268ef11e42139ecf487c90adb27a" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 38709 } }
I20260926 06:19:59.275244 6691 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4707268ef11e42139ecf487c90adb27a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:19:59.275296 6691 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4707268ef11e42139ecf487c90adb27a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4707268ef11e42139ecf487c90adb27a, State: Initialized, Role: FOLLOWER
I20260926 06:19:59.275444 6691 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4707268ef11e42139ecf487c90adb27a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4707268ef11e42139ecf487c90adb27a" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 38709 } }
I20260926 06:19:59.275535 6691 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4707268ef11e42139ecf487c90adb27a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:19:59.275578 6691 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4707268ef11e42139ecf487c90adb27a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:19:59.275624 6691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4707268ef11e42139ecf487c90adb27a [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:19:59.276127 6691 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4707268ef11e42139ecf487c90adb27a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4707268ef11e42139ecf487c90adb27a" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 38709 } }
I20260926 06:19:59.276252 6691 leader_election.cc:302] T 00000000000000000000000000000000 P 4707268ef11e42139ecf487c90adb27a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4707268ef11e42139ecf487c90adb27a; no voters:
I20260926 06:19:59.276453 6691 leader_election.cc:288] T 00000000000000000000000000000000 P 4707268ef11e42139ecf487c90adb27a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:19:59.276476 6696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4707268ef11e42139ecf487c90adb27a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:19:59.276589 6696 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4707268ef11e42139ecf487c90adb27a [term 1 LEADER]: Becoming Leader. State: Replica: 4707268ef11e42139ecf487c90adb27a, State: Running, Role: LEADER
I20260926 06:19:59.276711 6691 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4707268ef11e42139ecf487c90adb27a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:19:59.276741 6696 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4707268ef11e42139ecf487c90adb27a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4707268ef11e42139ecf487c90adb27a" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 38709 } }
I20260926 06:19:59.277485 6698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4707268ef11e42139ecf487c90adb27a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4707268ef11e42139ecf487c90adb27a. Latest consensus state: current_term: 1 leader_uuid: "4707268ef11e42139ecf487c90adb27a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4707268ef11e42139ecf487c90adb27a" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 38709 } } }
I20260926 06:19:59.277575 6698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4707268ef11e42139ecf487c90adb27a [sys.catalog]: This master's current role is: LEADER
I20260926 06:19:59.277974 6697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4707268ef11e42139ecf487c90adb27a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4707268ef11e42139ecf487c90adb27a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4707268ef11e42139ecf487c90adb27a" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 38709 } } }
I20260926 06:19:59.278043 6697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4707268ef11e42139ecf487c90adb27a [sys.catalog]: This master's current role is: LEADER
W20260926 06:19:59.278653 6712 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4707268ef11e42139ecf487c90adb27a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:19:59.278759 6712 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:19:59.278851 6705 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 26 06:19:59 dist-test-slave-pd0t krb5kdc[6600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:19:59.279026 6705 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:19:59.281116 6705 catalog_manager.cc:1347] Generated new cluster ID: 3ee1fdd338724904b7ca1907699fc248
I20260926 06:19:59.281227 6705 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:19:59.284130 6695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:59.272670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36526 (local address 127.4.202.190:38709)
0926 06:19:59.272859 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:59.272868 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:59.272902 (+ 34us) server_negotiation.cc:407] Connection header received
0926 06:19:59.273359 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:59.273365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:59.273412 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:59.273558 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:19:59.274168 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.275121 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.276028 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.276548 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.276928 (+ 380us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:59.279483 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:19:59.279487 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:19:59.279488 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:19:59.279501 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:19:59.282218 (+ 2717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:59.282641 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:59.282644 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:59.282645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:59.282736 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:59.282944 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:59.282945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:59.282945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:59.283180 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:19:59.283831 (+ 651us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:59.283887 (+ 56us) server_negotiation.cc:299] Negotiation successful
0926 06:19:59.283986 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":77,"threads_started":1}
I20260926 06:19:59.296027 6705 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:19:59.296602 6705 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:19:59.302933 6705 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4707268ef11e42139ecf487c90adb27a: Generated new TSK 0
I20260926 06:19:59.303133 6705 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.202.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
I20260926 06:19:59.331821 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.129:0
--local_ip_for_outbound_sockets=127.4.202.129
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:38709
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:37801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:19:59.412653 6717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:19:59.412874 6717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:19:59.412916 6717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:19:59.414052 6717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:19:59.414113 6717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:19:59.414149 6717 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 06:19:59.414196 6717 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 06:19:59.414232 6717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.129
I20260926 06:19:59.415759 6717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:37801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.202.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.202.190:38709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.129
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:19:59.416033 6717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:19:59.416255 6717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 06:19:59.421133 6717 server_base.cc:1034] running on GCE node
W20260926 06:19:59.421131 6723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:19:59.421156 6722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:19:59.421154 6725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:19:59.421535 6717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:19:59.421774 6717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:19:59.422935 6717 hybrid_clock.cc:648] HybridClock initialized: now 1790403599422931 us; error 37 us; skew 500 ppm
Sep 26 06:19:59 dist-test-slave-pd0t krb5kdc[6600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403599, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:19:59.425334 6717 init.cc:377] Logged in from keytab as kudu/127.4.202.129@KRBTEST.COM (short username kudu)
I20260926 06:19:59.425807 6717 webserver.cc:460] Webserver started at http://127.4.202.129:41639/ using document root <none> and password file <none>
I20260926 06:19:59.425989 6717 fs_manager.cc:359] Metadata directory not provided
I20260926 06:19:59.426031 6717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:19:59.426115 6717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:19:59.427166 6717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-0/data/instance:
uuid: "233d6dedde314417b8658dadb05c360b"
format_stamp: "Formatted at 2026-09-26 06:19:59 on dist-test-slave-pd0t"
server_key: "684f86dca6fca2f9d771b0e7f984f8c4"
server_key_iv: "6a77c1fa9871780758e5c39a82c723e6"
server_key_version: "encryptionkey@0"
I20260926 06:19:59.427512 6717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance:
uuid: "233d6dedde314417b8658dadb05c360b"
format_stamp: "Formatted at 2026-09-26 06:19:59 on dist-test-slave-pd0t"
server_key: "684f86dca6fca2f9d771b0e7f984f8c4"
server_key_iv: "6a77c1fa9871780758e5c39a82c723e6"
server_key_version: "encryptionkey@0"
I20260926 06:19:59.428815 6717 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:19:59.429564 6732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:59.429765 6717 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:19:59.429859 6717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-0/wal
uuid: "233d6dedde314417b8658dadb05c360b"
format_stamp: "Formatted at 2026-09-26 06:19:59 on dist-test-slave-pd0t"
server_key: "684f86dca6fca2f9d771b0e7f984f8c4"
server_key_iv: "6a77c1fa9871780758e5c39a82c723e6"
server_key_version: "encryptionkey@0"
I20260926 06:19:59.429908 6717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:19:59.462126 6717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:19:59.464501 6717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:19:59.464619 6717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:19:59.465087 6717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:19:59.465123 6717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:59.465149 6717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:19:59.465201 6717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:59.472262 6717 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.129:33965
I20260926 06:19:59.472322 6847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.129:33965 every 8 connection(s)
Sep 26 06:19:59 dist-test-slave-pd0t krb5kdc[6600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403599, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:19:59.472720 6717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
I20260926 06:19:59.475513 6695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:59.465673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:49399 (local address 127.4.202.190:38709)
0926 06:19:59.465729 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:59.465731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:59.465939 (+ 208us) server_negotiation.cc:407] Connection header received
0926 06:19:59.466467 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:59.466468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:59.466480 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:59.466558 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:19:59.467556 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.468375 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.470034 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.470376 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.470429 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:59.473234 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:19:59.473235 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:19:59.473235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:19:59.473241 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:19:59.474378 (+ 1137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:59.474811 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:59.474812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:59.474812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:59.474862 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:59.475055 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:59.475056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:59.475056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:59.475145 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:19:59.475188 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:59.475393 (+ 205us) server_negotiation.cc:299] Negotiation successful
0926 06:19:59.475453 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 06:19:59.475674 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:59.465661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:38709 (local address 127.4.202.129:49399)
0926 06:19:59.465822 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:59.465856 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:59.466364 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:59.466694 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:59.466699 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:59.467127 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:19:59.467448 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:59.467456 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.468479 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.468480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:59.469928 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:59.469931 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.470451 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.470452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:59.470636 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:59.471221 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:19:59.471233 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:19:59.473143 (+ 1910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:19:59.474447 (+ 1304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:59.474452 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:59.474454 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:59.474747 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:59.474920 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:59.474921 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:59.474921 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:59.475013 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:59.475193 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:19:59.475197 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:19:59.475302 (+ 105us) client_negotiation.cc:769] Sending connection context
0926 06:19:59.475374 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 06:19:59.475486 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
I20260926 06:19:59.476190 6848 heartbeater.cc:347] Connected to a master server at 127.4.202.190:38709
I20260926 06:19:59.476274 6848 heartbeater.cc:464] Registering TS with master...
I20260926 06:19:59.476461 6848 heartbeater.cc:511] Master 127.4.202.190:38709 requested a full tablet report, sending...
I20260926 06:19:59.476954 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 6717
I20260926 06:19:59.476966 6644 ts_manager.cc:194] Registered new tserver with Master: 233d6dedde314417b8658dadb05c360b (127.4.202.129:33965)
I20260926 06:19:59.477046 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance
I20260926 06:19:59.477255 4906 external_mini_cluster.cc:1428] Setting key 4265acf68cd688d3fd5b9acdd3aed2ee
I20260926 06:19:59.477824 6644 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.129@KRBTEST.COM'} at 127.4.202.129:49399
I20260926 06:19:59.483361 6854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:59.477295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:38709 (local address 127.4.202.129:45425)
0926 06:19:59.477465 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:59.477476 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:59.477541 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:59.477649 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:59.477650 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:59.477875 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:19:59.477955 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:59.477957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.478793 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.478795 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:59.479584 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:59.479586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.479922 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.479923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:59.479999 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:59.480444 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:19:59.480449 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:19:59.480716 (+ 267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:19:59.482426 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:59.482427 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:59.482427 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:59.482633 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:59.482828 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:59.482829 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:59.482829 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:59.482879 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:59.483145 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:19:59.483145 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:19:59.483181 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 06:19:59.483232 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 06:19:59.483301 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":106,"threads_started":1}
I20260926 06:19:59.483548 6695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:59.477341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:45425 (local address 127.4.202.190:38709)
0926 06:19:59.477380 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:59.477382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:59.477520 (+ 138us) server_negotiation.cc:407] Connection header received
0926 06:19:59.477588 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:59.477589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:59.477599 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:59.477646 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:19:59.478015 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.478682 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.479669 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.479848 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.479904 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:59.480809 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:19:59.480810 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:19:59.480811 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:19:59.480817 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:19:59.482342 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:59.482696 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:59.482698 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:59.482698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:59.482758 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:59.482941 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:59.482942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:59.482942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:59.483078 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:19:59.483153 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:59.483258 (+ 105us) server_negotiation.cc:299] Negotiation successful
0926 06:19:59.483337 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
WARNING: no policy specified for kudu/127.4.202.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
I20260926 06:19:59.501654 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.130:0
--local_ip_for_outbound_sockets=127.4.202.130
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:38709
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:37801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:19:59.579875 6857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:19:59.580090 6857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:19:59.580132 6857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:19:59.581254 6857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:19:59.581319 6857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:19:59.581354 6857 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 06:19:59.581382 6857 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 06:19:59.581414 6857 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.130
I20260926 06:19:59.582834 6857 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:37801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.202.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.202.190:38709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6857
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.130
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:19:59.583082 6857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:19:59.583303 6857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:19:59.588132 6862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:19:59.588249 6857 server_base.cc:1034] running on GCE node
W20260926 06:19:59.588215 6863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:19:59.588201 6865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:19:59.588666 6857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:19:59.588995 6857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:19:59.590163 6857 hybrid_clock.cc:648] HybridClock initialized: now 1790403599590146 us; error 32 us; skew 500 ppm
Sep 26 06:19:59 dist-test-slave-pd0t krb5kdc[6600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403599, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:19:59.592583 6857 init.cc:377] Logged in from keytab as kudu/127.4.202.130@KRBTEST.COM (short username kudu)
I20260926 06:19:59.592934 6857 webserver.cc:460] Webserver started at http://127.4.202.130:44043/ using document root <none> and password file <none>
I20260926 06:19:59.593106 6857 fs_manager.cc:359] Metadata directory not provided
I20260926 06:19:59.593143 6857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:19:59.593226 6857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:19:59.594149 6857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-1/data/instance:
uuid: "17bfeab63b3149a1bdedc53208a4bf57"
format_stamp: "Formatted at 2026-09-26 06:19:59 on dist-test-slave-pd0t"
server_key: "df652306e1b057a005b29d68b13dda55"
server_key_iv: "cec57a1f53cfcebba31500655bf3df6d"
server_key_version: "encryptionkey@0"
I20260926 06:19:59.594432 6857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance:
uuid: "17bfeab63b3149a1bdedc53208a4bf57"
format_stamp: "Formatted at 2026-09-26 06:19:59 on dist-test-slave-pd0t"
server_key: "df652306e1b057a005b29d68b13dda55"
server_key_iv: "cec57a1f53cfcebba31500655bf3df6d"
server_key_version: "encryptionkey@0"
I20260926 06:19:59.595772 6857 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:19:59.596455 6872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:59.596617 6857 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:19:59.596710 6857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-1/wal
uuid: "17bfeab63b3149a1bdedc53208a4bf57"
format_stamp: "Formatted at 2026-09-26 06:19:59 on dist-test-slave-pd0t"
server_key: "df652306e1b057a005b29d68b13dda55"
server_key_iv: "cec57a1f53cfcebba31500655bf3df6d"
server_key_version: "encryptionkey@0"
I20260926 06:19:59.596784 6857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:19:59.610211 6857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:19:59.611943 6857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:19:59.612102 6857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:19:59.612560 6857 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:19:59.612599 6857 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:59.612629 6857 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:19:59.612673 6857 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 06:19:59 dist-test-slave-pd0t krb5kdc[6600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403599, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:19:59.619544 6857 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.130:43075
I20260926 06:19:59.619652 6987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.130:43075 every 8 connection(s)
I20260926 06:19:59.619910 6857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
I20260926 06:19:59.622536 6695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:59.613037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:37527 (local address 127.4.202.190:38709)
0926 06:19:59.613100 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:59.613102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:59.613327 (+ 225us) server_negotiation.cc:407] Connection header received
0926 06:19:59.613796 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:59.613798 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:59.613811 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:59.613868 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:19:59.614807 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.615605 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.617218 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.617530 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.617598 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:59.620044 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:19:59.620045 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:19:59.620046 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:19:59.620053 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:19:59.621531 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:59.621872 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:59.621872 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:59.621872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:59.621918 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:59.622085 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:59.622086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:59.622086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:59.622176 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:19:59.622237 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:59.622423 (+ 186us) server_negotiation.cc:299] Negotiation successful
0926 06:19:59.622476 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 06:19:59.622762 6881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:59.613027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:38709 (local address 127.4.202.130:37527)
0926 06:19:59.613202 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:59.613250 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:59.613688 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:59.613984 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:59.613989 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:59.614378 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:19:59.614699 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:59.614706 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.615700 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.615701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:59.617113 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:59.617116 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.617605 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.617606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:59.617761 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:59.618251 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:19:59.618259 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:19:59.619941 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:19:59.621605 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:59.621609 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:59.621610 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:59.621807 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:59.621959 (+ 152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:59.621959 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:59.621959 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:59.622037 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:59.622233 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:19:59.622237 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:19:59.622337 (+ 100us) client_negotiation.cc:769] Sending connection context
0926 06:19:59.622412 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 06:19:59.622516 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":97,"threads_started":1}
I20260926 06:19:59.623214 6988 heartbeater.cc:347] Connected to a master server at 127.4.202.190:38709
I20260926 06:19:59.623291 6988 heartbeater.cc:464] Registering TS with master...
I20260926 06:19:59.623484 6988 heartbeater.cc:511] Master 127.4.202.190:38709 requested a full tablet report, sending...
I20260926 06:19:59.623809 6645 ts_manager.cc:194] Registered new tserver with Master: 17bfeab63b3149a1bdedc53208a4bf57 (127.4.202.130:43075)
I20260926 06:19:59.624192 6645 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.130@KRBTEST.COM'} at 127.4.202.130:37527
I20260926 06:19:59.625823 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 6857
I20260926 06:19:59.625922 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance
I20260926 06:19:59.626092 4906 external_mini_cluster.cc:1428] Setting key f54f092ccb9a7d8a2f98b7429b17f07f
I20260926 06:19:59.629834 6695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:59.624264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:40377 (local address 127.4.202.190:38709)
0926 06:19:59.624303 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:59.624305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:59.624386 (+ 81us) server_negotiation.cc:407] Connection header received
0926 06:19:59.624448 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:59.624448 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:59.624455 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:59.624516 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:19:59.624835 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.625285 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.626184 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.626384 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.626463 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:59.627354 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:19:59.627356 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:19:59.627356 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:19:59.627363 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:19:59.628690 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:59.629035 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:59.629036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:59.629037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:59.629094 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:59.629300 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:59.629301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:59.629301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:59.629443 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:19:59.629521 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:59.629685 (+ 164us) server_negotiation.cc:299] Negotiation successful
0926 06:19:59.629751 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 06:19:59.629843 6994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:59.624203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:38709 (local address 127.4.202.130:40377)
0926 06:19:59.624332 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:59.624342 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:59.624398 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:59.624548 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:59.624549 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:59.624707 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:19:59.624754 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:59.624755 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.625374 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.625375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:59.626068 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:59.626070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.626463 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.626464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:59.626549 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:59.627011 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:19:59.627016 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:19:59.627265 (+ 249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:19:59.628793 (+ 1528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:59.628794 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:59.628795 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:59.628960 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:59.629164 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:59.629165 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:59.629165 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:59.629222 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:59.629521 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:19:59.629522 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:19:59.629566 (+ 44us) client_negotiation.cc:769] Sending connection context
0926 06:19:59.629648 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 06:19:59.629749 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":66,"threads_started":1}
WARNING: no policy specified for kudu/127.4.202.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
I20260926 06:19:59.649452 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.131:0
--local_ip_for_outbound_sockets=127.4.202.131
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:38709
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:37801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:19:59.728325 6997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:19:59.728513 6997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:19:59.728536 6997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:19:59.729557 6997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:19:59.729588 6997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:19:59.729609 6997 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 06:19:59.729620 6997 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 06:19:59.729636 6997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.131
I20260926 06:19:59.730974 6997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:37801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.202.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.202.190:38709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.131
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:19:59.731146 6997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:19:59.731297 6997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:19:59.736099 7003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:19:59.736111 7005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:19:59.736109 7002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:19:59.736338 6997 server_base.cc:1034] running on GCE node
I20260926 06:19:59.736523 6997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:19:59.736750 6997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:19:59.737907 6997 hybrid_clock.cc:648] HybridClock initialized: now 1790403599737877 us; error 43 us; skew 500 ppm
Sep 26 06:19:59 dist-test-slave-pd0t krb5kdc[6600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403599, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:19:59.745147 6997 init.cc:377] Logged in from keytab as kudu/127.4.202.131@KRBTEST.COM (short username kudu)
I20260926 06:19:59.745577 6997 webserver.cc:460] Webserver started at http://127.4.202.131:35617/ using document root <none> and password file <none>
I20260926 06:19:59.745790 6997 fs_manager.cc:359] Metadata directory not provided
I20260926 06:19:59.745833 6997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:19:59.745990 6997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:19:59.747156 6997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-2/data/instance:
uuid: "0bab59c2bae246d3a307d3d8af888422"
format_stamp: "Formatted at 2026-09-26 06:19:59 on dist-test-slave-pd0t"
server_key: "0b1f0704ead0f9601ad7721f417052bd"
server_key_iv: "eb294358df509b14d13710b4298a4f47"
server_key_version: "encryptionkey@0"
I20260926 06:19:59.747498 6997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance:
uuid: "0bab59c2bae246d3a307d3d8af888422"
format_stamp: "Formatted at 2026-09-26 06:19:59 on dist-test-slave-pd0t"
server_key: "0b1f0704ead0f9601ad7721f417052bd"
server_key_iv: "eb294358df509b14d13710b4298a4f47"
server_key_version: "encryptionkey@0"
I20260926 06:19:59.748837 6997 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:19:59.749529 7012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:59.749714 6997 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:19:59.749794 6997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-2/wal
uuid: "0bab59c2bae246d3a307d3d8af888422"
format_stamp: "Formatted at 2026-09-26 06:19:59 on dist-test-slave-pd0t"
server_key: "0b1f0704ead0f9601ad7721f417052bd"
server_key_iv: "eb294358df509b14d13710b4298a4f47"
server_key_version: "encryptionkey@0"
I20260926 06:19:59.749845 6997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:19:59.787768 6997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:19:59.789775 6997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:19:59.789883 6997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:19:59.790383 6997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:19:59.790426 6997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:59.790449 6997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:19:59.790491 6997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:19:59.797163 6997 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.131:39917
I20260926 06:19:59.797235 7127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.131:39917 every 8 connection(s)
Sep 26 06:19:59 dist-test-slave-pd0t krb5kdc[6600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403599, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:19:59.797578 6997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
I20260926 06:19:59.800788 6695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:59.790883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:35613 (local address 127.4.202.190:38709)
0926 06:19:59.790947 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:59.790950 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:59.791138 (+ 188us) server_negotiation.cc:407] Connection header received
0926 06:19:59.791619 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:59.791621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:59.791634 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:59.791719 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:19:59.792528 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.793394 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.794841 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.795198 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.795285 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:59.797851 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:19:59.797853 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:19:59.797853 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:19:59.797861 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:19:59.799387 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:59.799822 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:59.799824 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:59.799824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:59.799921 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:59.800170 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:59.800170 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:59.800171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:59.800295 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:19:59.800365 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:59.800650 (+ 285us) server_negotiation.cc:299] Negotiation successful
0926 06:19:59.800722 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 06:19:59.800897 7021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:59.790851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:38709 (local address 127.4.202.131:35613)
0926 06:19:59.791002 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:59.791053 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:59.791525 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:59.791804 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:59.791808 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:59.792142 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:19:59.792412 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:59.792418 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.793507 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.793508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:59.794776 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:59.794778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.795299 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.795300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:59.795404 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:59.795914 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:19:59.795932 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:19:59.797754 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:19:59.799459 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:59.799465 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:59.799467 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:59.799722 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:59.800000 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:59.800001 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:59.800001 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:59.800108 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:59.800393 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:19:59.800398 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:19:59.800509 (+ 111us) client_negotiation.cc:769] Sending connection context
0926 06:19:59.800607 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 06:19:59.800723 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":87,"threads_started":1}
I20260926 06:19:59.801343 7128 heartbeater.cc:347] Connected to a master server at 127.4.202.190:38709
I20260926 06:19:59.801432 7128 heartbeater.cc:464] Registering TS with master...
I20260926 06:19:59.801636 7128 heartbeater.cc:511] Master 127.4.202.190:38709 requested a full tablet report, sending...
I20260926 06:19:59.802057 6644 ts_manager.cc:194] Registered new tserver with Master: 0bab59c2bae246d3a307d3d8af888422 (127.4.202.131:39917)
I20260926 06:19:59.802572 6644 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.131@KRBTEST.COM'} at 127.4.202.131:35613
I20260926 06:19:59.803870 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 6997
I20260926 06:19:59.803958 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance
I20260926 06:19:59.804147 4906 external_mini_cluster.cc:1428] Setting key 21352d2ec0fad34a30fd58356b5a7897
I20260926 06:19:59.804971 4906 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:19:59.809518 6695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:59.802466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:40833 (local address 127.4.202.190:38709)
0926 06:19:59.802530 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:59.802532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:59.802780 (+ 248us) server_negotiation.cc:407] Connection header received
0926 06:19:59.802866 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:59.802867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:59.802884 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:59.802954 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:19:59.803744 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.804431 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.805409 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.805636 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.805727 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:59.806860 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:19:59.806861 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:19:59.806861 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:19:59.806867 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:19:59.808234 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:59.808701 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:59.808702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:59.808702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:59.808768 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:59.808982 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:59.808983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:59.808983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:59.809101 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:19:59.809184 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:59.809346 (+ 162us) server_negotiation.cc:299] Negotiation successful
0926 06:19:59.809423 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 06:19:59.809526 7134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:59.802432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:38709 (local address 127.4.202.131:40833)
0926 06:19:59.802685 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:59.802717 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:59.802808 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:59.803399 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:59.803400 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:59.803600 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:19:59.803650 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:59.803652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.804533 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.804534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:59.805310 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:59.805312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.805729 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.805730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:59.805808 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:59.806289 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:19:59.806294 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:19:59.806774 (+ 480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:19:59.808344 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:59.808345 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:59.808345 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:59.808611 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:59.808839 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:19:59.808840 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:19:59.808841 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:19:59.808903 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:19:59.809176 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:19:59.809177 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:19:59.809224 (+ 47us) client_negotiation.cc:769] Sending connection context
0926 06:19:59.809320 (+ 96us) client_negotiation.cc:240] Negotiation successful
0926 06:19:59.809425 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":199,"threads_started":1}
Sep 26 06:19:59 dist-test-slave-pd0t krb5kdc[6600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 06:19:59 dist-test-slave-pd0t krb5kdc[6600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:19:59.828080 6695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:59.821301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36530 (local address 127.4.202.190:38709)
0926 06:19:59.821360 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:59.821362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:59.821371 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:19:59.821425 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:59.821426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:59.821435 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:59.821505 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:19:59.822048 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.822496 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.823427 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.823613 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.823695 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:59.825727 (+ 2032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:19:59.825728 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:19:59.825728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:19:59.825735 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:19:59.826970 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:59.827340 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:59.827342 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:59.827342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:59.827396 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:19:59.827599 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:19:59.827600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:19:59.827600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:19:59.827702 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:19:59.827801 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:59.827917 (+ 116us) server_negotiation.cc:299] Negotiation successful
0926 06:19:59.827985 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 06:19:59.829656 6644 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36530:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:19:59.831429 6644 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:19:59.837947 7147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:59.833836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:43075 (local address 127.0.0.1:42460)
0926 06:19:59.834200 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:59.834247 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:59.834330 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:59.834548 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:59.834551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:59.834554 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:19:59.834675 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:59.834678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.835647 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.835648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:59.836701 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:59.836704 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.837637 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.837638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:59.837721 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:59.837736 (+ 15us) client_negotiation.cc:769] Sending connection context
0926 06:19:59.837819 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 06:19:59.837872 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":50,"threads_started":1}
I20260926 06:19:59.837980 7148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:59.834007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42460 (local address 127.4.202.130:43075)
0926 06:19:59.834160 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:59.834165 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:59.834321 (+ 156us) server_negotiation.cc:407] Connection header received
0926 06:19:59.834418 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:59.834421 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:59.834438 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:59.834524 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:19:59.834748 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.835542 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.836796 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.837548 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.837621 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:59.837675 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:59.837841 (+ 166us) server_negotiation.cc:299] Negotiation successful
0926 06:19:59.837880 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":94,"threads_started":1}
I20260926 06:19:59.838481 7149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:59.834043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58658 (local address 127.4.202.129:33965)
0926 06:19:59.834262 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:59.834268 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:59.834428 (+ 160us) server_negotiation.cc:407] Connection header received
0926 06:19:59.834463 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:59.834465 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:59.834484 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:59.834628 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:19:59.834840 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.836092 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.837346 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.838120 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.838194 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:59.838246 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:59.838343 (+ 97us) server_negotiation.cc:299] Negotiation successful
0926 06:19:59.838378 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":94,"threads_started":1}
I20260926 06:19:59.838471 7145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:59.833836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:33965 (local address 127.0.0.1:58658)
0926 06:19:59.834235 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:59.834247 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:59.834331 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:59.834700 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:59.834701 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:59.834703 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:19:59.834753 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:59.834754 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.836197 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.836198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:59.837253 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:59.837255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.838204 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.838205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:59.838269 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:59.838273 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:19:59.838330 (+ 57us) client_negotiation.cc:240] Negotiation successful
0926 06:19:59.838370 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"mutex_wait_us":3,"thread_start_us":93,"threads_started":1}
I20260926 06:19:59.838495 7146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:59.833844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:39917 (local address 127.0.0.1:54084)
0926 06:19:59.834441 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0926 06:19:59.834451 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:19:59.834504 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:19:59.834918 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:19:59.834919 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:19:59.834922 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:19:59.834968 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:59.834970 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.836254 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.836256 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:59.837226 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:19:59.837228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.838222 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.838223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:19:59.838288 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:59.838292 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:19:59.838345 (+ 53us) client_negotiation.cc:240] Negotiation successful
0926 06:19:59.838398 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":96,"threads_started":1}
I20260926 06:19:59.838775 7150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:19:59.834068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54084 (local address 127.4.202.131:39917)
0926 06:19:59.834336 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0926 06:19:59.834340 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:19:59.834766 (+ 426us) server_negotiation.cc:407] Connection header received
0926 06:19:59.834808 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:19:59.834810 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:19:59.834828 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:19:59.834921 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:19:59.835037 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.836182 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.837330 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:19:59.838050 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:19:59.838130 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:19:59.838187 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 06:19:59.838531 (+ 344us) server_negotiation.cc:299] Negotiation successful
0926 06:19:59.838686 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":78,"threads_started":1}
I20260926 06:19:59.839025 6922 tablet_service.cc:1467] Processing CreateTablet for tablet d0c11e6cd4e6478d8df8b54e58315e4f (DEFAULT_TABLE table=test-table [id=e40eeceffd3c49b1a3de63d5c30ddd22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:19:59.839442 6922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0c11e6cd4e6478d8df8b54e58315e4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:19:59.839479 6782 tablet_service.cc:1467] Processing CreateTablet for tablet d0c11e6cd4e6478d8df8b54e58315e4f (DEFAULT_TABLE table=test-table [id=e40eeceffd3c49b1a3de63d5c30ddd22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:19:59.839815 6782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0c11e6cd4e6478d8df8b54e58315e4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:19:59.842716 7151 tablet_bootstrap.cc:492] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57: Bootstrap starting.
I20260926 06:19:59.843176 7062 tablet_service.cc:1467] Processing CreateTablet for tablet d0c11e6cd4e6478d8df8b54e58315e4f (DEFAULT_TABLE table=test-table [id=e40eeceffd3c49b1a3de63d5c30ddd22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:19:59.843385 7151 tablet_bootstrap.cc:654] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57: Neither blocks nor log segments found. Creating new log.
I20260926 06:19:59.843509 7062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0c11e6cd4e6478d8df8b54e58315e4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:19:59.843768 7151 log.cc:824] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57: Log is configured to *not* fsync() on all Append() calls
I20260926 06:19:59.844596 7151 tablet_bootstrap.cc:492] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57: No bootstrap required, opened a new log
I20260926 06:19:59.844674 7151 ts_tablet_manager.cc:1397] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:19:59.846854 7152 tablet_bootstrap.cc:492] T d0c11e6cd4e6478d8df8b54e58315e4f P 233d6dedde314417b8658dadb05c360b: Bootstrap starting.
I20260926 06:19:59.846968 7151 raft_consensus.cc:348] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:19:59.847146 7151 raft_consensus.cc:374] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:19:59.847196 7151 raft_consensus.cc:729] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17bfeab63b3149a1bdedc53208a4bf57, State: Initialized, Role: FOLLOWER
I20260926 06:19:59.847333 7151 consensus_queue.cc:260] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:19:59.847576 7152 tablet_bootstrap.cc:654] T d0c11e6cd4e6478d8df8b54e58315e4f P 233d6dedde314417b8658dadb05c360b: Neither blocks nor log segments found. Creating new log.
I20260926 06:19:59.847615 7151 ts_tablet_manager.cc:1428] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 06:19:59.847656 6988 heartbeater.cc:503] Master 127.4.202.190:38709 was elected leader, sending a full tablet report...
I20260926 06:19:59.847800 7154 tablet_bootstrap.cc:492] T d0c11e6cd4e6478d8df8b54e58315e4f P 0bab59c2bae246d3a307d3d8af888422: Bootstrap starting.
I20260926 06:19:59.847913 7152 log.cc:824] T d0c11e6cd4e6478d8df8b54e58315e4f P 233d6dedde314417b8658dadb05c360b: Log is configured to *not* fsync() on all Append() calls
I20260926 06:19:59.848836 7154 tablet_bootstrap.cc:654] T d0c11e6cd4e6478d8df8b54e58315e4f P 0bab59c2bae246d3a307d3d8af888422: Neither blocks nor log segments found. Creating new log.
I20260926 06:19:59.848942 7152 tablet_bootstrap.cc:492] T d0c11e6cd4e6478d8df8b54e58315e4f P 233d6dedde314417b8658dadb05c360b: No bootstrap required, opened a new log
I20260926 06:19:59.849005 7152 ts_tablet_manager.cc:1397] T d0c11e6cd4e6478d8df8b54e58315e4f P 233d6dedde314417b8658dadb05c360b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:19:59.849169 7154 log.cc:824] T d0c11e6cd4e6478d8df8b54e58315e4f P 0bab59c2bae246d3a307d3d8af888422: Log is configured to *not* fsync() on all Append() calls
I20260926 06:19:59.849912 7154 tablet_bootstrap.cc:492] T d0c11e6cd4e6478d8df8b54e58315e4f P 0bab59c2bae246d3a307d3d8af888422: No bootstrap required, opened a new log
I20260926 06:19:59.849989 7154 ts_tablet_manager.cc:1397] T d0c11e6cd4e6478d8df8b54e58315e4f P 0bab59c2bae246d3a307d3d8af888422: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:19:59.850703 7152 raft_consensus.cc:348] T d0c11e6cd4e6478d8df8b54e58315e4f P 233d6dedde314417b8658dadb05c360b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:19:59.850840 7152 raft_consensus.cc:374] T d0c11e6cd4e6478d8df8b54e58315e4f P 233d6dedde314417b8658dadb05c360b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:19:59.850881 7152 raft_consensus.cc:729] T d0c11e6cd4e6478d8df8b54e58315e4f P 233d6dedde314417b8658dadb05c360b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233d6dedde314417b8658dadb05c360b, State: Initialized, Role: FOLLOWER
I20260926 06:19:59.851014 7152 consensus_queue.cc:260] T d0c11e6cd4e6478d8df8b54e58315e4f P 233d6dedde314417b8658dadb05c360b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:19:59.851306 7152 ts_tablet_manager.cc:1428] T d0c11e6cd4e6478d8df8b54e58315e4f P 233d6dedde314417b8658dadb05c360b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 06:19:59.851358 6848 heartbeater.cc:503] Master 127.4.202.190:38709 was elected leader, sending a full tablet report...
I20260926 06:19:59.851943 7154 raft_consensus.cc:348] T d0c11e6cd4e6478d8df8b54e58315e4f P 0bab59c2bae246d3a307d3d8af888422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:19:59.852144 7154 raft_consensus.cc:374] T d0c11e6cd4e6478d8df8b54e58315e4f P 0bab59c2bae246d3a307d3d8af888422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:19:59.852200 7154 raft_consensus.cc:729] T d0c11e6cd4e6478d8df8b54e58315e4f P 0bab59c2bae246d3a307d3d8af888422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bab59c2bae246d3a307d3d8af888422, State: Initialized, Role: FOLLOWER
I20260926 06:19:59.852365 7154 consensus_queue.cc:260] T d0c11e6cd4e6478d8df8b54e58315e4f P 0bab59c2bae246d3a307d3d8af888422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:19:59.852632 7154 ts_tablet_manager.cc:1428] T d0c11e6cd4e6478d8df8b54e58315e4f P 0bab59c2bae246d3a307d3d8af888422: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 06:19:59.852684 7128 heartbeater.cc:503] Master 127.4.202.190:38709 was elected leader, sending a full tablet report...
W20260926 06:19:59.870488 6989 tablet.cc:2369] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 06:19:59.973385 6849 tablet.cc:2369] T d0c11e6cd4e6478d8df8b54e58315e4f P 233d6dedde314417b8658dadb05c360b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 06:20:00.048404 7129 tablet.cc:2369] T d0c11e6cd4e6478d8df8b54e58315e4f P 0bab59c2bae246d3a307d3d8af888422: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:20:00.052457 7155 raft_consensus.cc:482] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:20:00.052613 7155 raft_consensus.cc:504] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:00.052953 7155 leader_election.cc:288] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233d6dedde314417b8658dadb05c360b (127.4.202.129:33965), 0bab59c2bae246d3a307d3d8af888422 (127.4.202.131:39917)
I20260926 06:20:00.055948 7149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:00.053209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:38563 (local address 127.4.202.129:33965)
0926 06:20:00.053254 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:00.053257 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:00.053268 (+ 11us) server_negotiation.cc:407] Connection header received
0926 06:20:00.053300 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:00.053301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:00.053311 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:00.053397 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:20:00.053635 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.054168 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.055232 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.055594 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.055646 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:00.055653 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:00.055834 (+ 181us) server_negotiation.cc:299] Negotiation successful
0926 06:20:00.055876 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 06:20:00.055953 7150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:00.053209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:38909 (local address 127.4.202.131:39917)
0926 06:20:00.053258 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:00.053260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:00.053435 (+ 175us) server_negotiation.cc:407] Connection header received
0926 06:20:00.053463 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:00.053464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:00.053474 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:00.053545 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:20:00.053735 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.054183 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.055222 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.055596 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.055692 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:00.055699 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:00.055860 (+ 161us) server_negotiation.cc:299] Negotiation successful
0926 06:20:00.055894 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 06:20:00.055953 6881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:00.053147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:33965 (local address 127.4.202.130:38563)
0926 06:20:00.053185 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:00.053196 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:00.053241 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:00.053457 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:00.053458 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:00.053460 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:20:00.053537 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:00.053539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.054256 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.054257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:00.055136 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:00.055138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.055676 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.055678 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:00.055764 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:00.055767 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:20:00.055809 (+ 42us) client_negotiation.cc:240] Negotiation successful
0926 06:20:00.055867 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19,"mutex_wait_us":3}
I20260926 06:20:00.055972 7160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:00.053147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:39917 (local address 127.4.202.130:38909)
0926 06:20:00.053338 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:00.053348 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:00.053409 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:00.053552 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:00.053553 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:00.053554 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:20:00.053657 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:00.053659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.054292 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.054293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:00.055131 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:00.055134 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.055711 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.055711 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:00.055771 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:00.055774 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:20:00.055846 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 06:20:00.055879 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":47,"thread_start_us":87,"threads_started":1}
I20260926 06:20:00.056296 6802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0c11e6cd4e6478d8df8b54e58315e4f" candidate_uuid: "17bfeab63b3149a1bdedc53208a4bf57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233d6dedde314417b8658dadb05c360b" is_pre_election: true
I20260926 06:20:00.056303 7082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0c11e6cd4e6478d8df8b54e58315e4f" candidate_uuid: "17bfeab63b3149a1bdedc53208a4bf57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bab59c2bae246d3a307d3d8af888422" is_pre_election: true
I20260926 06:20:00.056444 7082 raft_consensus.cc:2454] T d0c11e6cd4e6478d8df8b54e58315e4f P 0bab59c2bae246d3a307d3d8af888422 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17bfeab63b3149a1bdedc53208a4bf57 in term 0.
I20260926 06:20:00.056444 6802 raft_consensus.cc:2454] T d0c11e6cd4e6478d8df8b54e58315e4f P 233d6dedde314417b8658dadb05c360b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17bfeab63b3149a1bdedc53208a4bf57 in term 0.
I20260926 06:20:00.056679 6876 leader_election.cc:302] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bab59c2bae246d3a307d3d8af888422, 17bfeab63b3149a1bdedc53208a4bf57; no voters:
I20260926 06:20:00.056864 7155 raft_consensus.cc:2781] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:20:00.056936 7155 raft_consensus.cc:482] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:20:00.056968 7155 raft_consensus.cc:3037] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:00.057888 7155 raft_consensus.cc:504] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:00.058065 7155 leader_election.cc:288] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57 [CANDIDATE]: Term 1 election: Requested vote from peers 233d6dedde314417b8658dadb05c360b (127.4.202.129:33965), 0bab59c2bae246d3a307d3d8af888422 (127.4.202.131:39917)
I20260926 06:20:00.058214 6802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0c11e6cd4e6478d8df8b54e58315e4f" candidate_uuid: "17bfeab63b3149a1bdedc53208a4bf57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233d6dedde314417b8658dadb05c360b"
I20260926 06:20:00.058259 7082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0c11e6cd4e6478d8df8b54e58315e4f" candidate_uuid: "17bfeab63b3149a1bdedc53208a4bf57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bab59c2bae246d3a307d3d8af888422"
I20260926 06:20:00.058298 6802 raft_consensus.cc:3037] T d0c11e6cd4e6478d8df8b54e58315e4f P 233d6dedde314417b8658dadb05c360b [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:00.058324 7082 raft_consensus.cc:3037] T d0c11e6cd4e6478d8df8b54e58315e4f P 0bab59c2bae246d3a307d3d8af888422 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:00.059171 6802 raft_consensus.cc:2454] T d0c11e6cd4e6478d8df8b54e58315e4f P 233d6dedde314417b8658dadb05c360b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17bfeab63b3149a1bdedc53208a4bf57 in term 1.
I20260926 06:20:00.059171 7082 raft_consensus.cc:2454] T d0c11e6cd4e6478d8df8b54e58315e4f P 0bab59c2bae246d3a307d3d8af888422 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17bfeab63b3149a1bdedc53208a4bf57 in term 1.
I20260926 06:20:00.059338 6876 leader_election.cc:302] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bab59c2bae246d3a307d3d8af888422, 17bfeab63b3149a1bdedc53208a4bf57; no voters:
I20260926 06:20:00.059432 7155 raft_consensus.cc:2781] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:20:00.059578 7155 raft_consensus.cc:686] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57 [term 1 LEADER]: Becoming Leader. State: Replica: 17bfeab63b3149a1bdedc53208a4bf57, State: Running, Role: LEADER
I20260926 06:20:00.059643 7155 consensus_queue.cc:237] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:00.060428 6644 catalog_manager.cc:5495] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 17bfeab63b3149a1bdedc53208a4bf57 (127.4.202.130). New cstate: current_term: 1 leader_uuid: "17bfeab63b3149a1bdedc53208a4bf57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } health_report { overall_health: UNKNOWN } } }
Sep 26 06:20:00 dist-test-slave-pd0t krb5kdc[6600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403599, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:20:00.161568 7168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:00.155192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:38709 (local address 127.0.0.1:36534)
0926 06:20:00.155329 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:00.155338 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:00.155396 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:00.155536 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:00.155537 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:00.155849 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:00.155900 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:00.155902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.156500 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.156500 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:00.157038 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:00.157039 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.157344 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.157345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:00.157383 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:00.157852 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:00.157856 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:00.159203 (+ 1347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:00.160650 (+ 1447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:00.160657 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:00.160659 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:00.160841 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:00.161015 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:00.161016 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:00.161016 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:00.161073 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:00.161279 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:00.161284 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:00.161322 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 06:20:00.161375 (+ 53us) client_negotiation.cc:240] Negotiation successful
0926 06:20:00.161469 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":67,"threads_started":1}
I20260926 06:20:00.161608 6695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:00.155258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36534 (local address 127.4.202.190:38709)
0926 06:20:00.155357 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:00.155358 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:00.155382 (+ 24us) server_negotiation.cc:407] Connection header received
0926 06:20:00.155455 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:00.155456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:00.155465 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:00.155516 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:00.155969 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.156434 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.157113 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.157280 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.157329 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:00.159279 (+ 1950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:00.159280 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:00.159281 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:00.159286 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:00.160577 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:00.160905 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:00.160906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:00.160906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:00.160952 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:00.161127 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:00.161128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:00.161128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:00.161220 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:00.161269 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:00.161417 (+ 148us) server_negotiation.cc:299] Negotiation successful
0926 06:20:00.161512 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":69}
I20260926 06:20:00.163736 6644 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:36534:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 06:20:00.164147 6644 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:20:00.165789 6922 tablet_service.cc:1467] Processing CreateTablet for tablet f74cdeb57897458da52df52465769a23 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7d417d95b26b454dbe92368325b284e9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 06:20:00.165812 6782 tablet_service.cc:1467] Processing CreateTablet for tablet f74cdeb57897458da52df52465769a23 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7d417d95b26b454dbe92368325b284e9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 06:20:00.165810 7062 tablet_service.cc:1467] Processing CreateTablet for tablet f74cdeb57897458da52df52465769a23 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7d417d95b26b454dbe92368325b284e9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 06:20:00.165977 6922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f74cdeb57897458da52df52465769a23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:00.165982 7062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f74cdeb57897458da52df52465769a23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:00.165982 6782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f74cdeb57897458da52df52465769a23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:00.167290 7151 tablet_bootstrap.cc:492] T f74cdeb57897458da52df52465769a23 P 17bfeab63b3149a1bdedc53208a4bf57: Bootstrap starting.
I20260926 06:20:00.167577 7152 tablet_bootstrap.cc:492] T f74cdeb57897458da52df52465769a23 P 233d6dedde314417b8658dadb05c360b: Bootstrap starting.
I20260926 06:20:00.167639 7154 tablet_bootstrap.cc:492] T f74cdeb57897458da52df52465769a23 P 0bab59c2bae246d3a307d3d8af888422: Bootstrap starting.
I20260926 06:20:00.167702 7151 tablet_bootstrap.cc:654] T f74cdeb57897458da52df52465769a23 P 17bfeab63b3149a1bdedc53208a4bf57: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:00.168102 7152 tablet_bootstrap.cc:654] T f74cdeb57897458da52df52465769a23 P 233d6dedde314417b8658dadb05c360b: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:00.168367 7154 tablet_bootstrap.cc:654] T f74cdeb57897458da52df52465769a23 P 0bab59c2bae246d3a307d3d8af888422: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:00.168809 7152 tablet_bootstrap.cc:492] T f74cdeb57897458da52df52465769a23 P 233d6dedde314417b8658dadb05c360b: No bootstrap required, opened a new log
I20260926 06:20:00.168885 7152 ts_tablet_manager.cc:1397] T f74cdeb57897458da52df52465769a23 P 233d6dedde314417b8658dadb05c360b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 06:20:00.169046 7154 tablet_bootstrap.cc:492] T f74cdeb57897458da52df52465769a23 P 0bab59c2bae246d3a307d3d8af888422: No bootstrap required, opened a new log
I20260926 06:20:00.169103 7154 ts_tablet_manager.cc:1397] T f74cdeb57897458da52df52465769a23 P 0bab59c2bae246d3a307d3d8af888422: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 06:20:00.169065 7152 raft_consensus.cc:348] T f74cdeb57897458da52df52465769a23 P 233d6dedde314417b8658dadb05c360b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:00.169142 7152 raft_consensus.cc:374] T f74cdeb57897458da52df52465769a23 P 233d6dedde314417b8658dadb05c360b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:00.169181 7152 raft_consensus.cc:729] T f74cdeb57897458da52df52465769a23 P 233d6dedde314417b8658dadb05c360b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233d6dedde314417b8658dadb05c360b, State: Initialized, Role: FOLLOWER
I20260926 06:20:00.169238 7152 consensus_queue.cc:260] T f74cdeb57897458da52df52465769a23 P 233d6dedde314417b8658dadb05c360b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:00.169258 7154 raft_consensus.cc:348] T f74cdeb57897458da52df52465769a23 P 0bab59c2bae246d3a307d3d8af888422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:00.169322 7154 raft_consensus.cc:374] T f74cdeb57897458da52df52465769a23 P 0bab59c2bae246d3a307d3d8af888422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:00.169348 7154 raft_consensus.cc:729] T f74cdeb57897458da52df52465769a23 P 0bab59c2bae246d3a307d3d8af888422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bab59c2bae246d3a307d3d8af888422, State: Initialized, Role: FOLLOWER
I20260926 06:20:00.169364 7152 ts_tablet_manager.cc:1428] T f74cdeb57897458da52df52465769a23 P 233d6dedde314417b8658dadb05c360b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 06:20:00.169400 7154 consensus_queue.cc:260] T f74cdeb57897458da52df52465769a23 P 0bab59c2bae246d3a307d3d8af888422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:00.169425 7158 tablet_replica.cc:439] T f74cdeb57897458da52df52465769a23 P 233d6dedde314417b8658dadb05c360b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } } }
I20260926 06:20:00.169498 7154 ts_tablet_manager.cc:1428] T f74cdeb57897458da52df52465769a23 P 0bab59c2bae246d3a307d3d8af888422: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 06:20:00.169515 7158 tablet_replica.cc:442] T f74cdeb57897458da52df52465769a23 P 233d6dedde314417b8658dadb05c360b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:20:00.169551 7159 tablet_replica.cc:439] T f74cdeb57897458da52df52465769a23 P 0bab59c2bae246d3a307d3d8af888422: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } } }
I20260926 06:20:00.169641 7159 tablet_replica.cc:442] T f74cdeb57897458da52df52465769a23 P 0bab59c2bae246d3a307d3d8af888422: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:20:00.170032 7151 tablet_bootstrap.cc:492] T f74cdeb57897458da52df52465769a23 P 17bfeab63b3149a1bdedc53208a4bf57: No bootstrap required, opened a new log
I20260926 06:20:00.170102 7151 ts_tablet_manager.cc:1397] T f74cdeb57897458da52df52465769a23 P 17bfeab63b3149a1bdedc53208a4bf57: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260926 06:20:00.170248 7151 raft_consensus.cc:348] T f74cdeb57897458da52df52465769a23 P 17bfeab63b3149a1bdedc53208a4bf57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:00.170311 7151 raft_consensus.cc:374] T f74cdeb57897458da52df52465769a23 P 17bfeab63b3149a1bdedc53208a4bf57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:00.170336 7151 raft_consensus.cc:729] T f74cdeb57897458da52df52465769a23 P 17bfeab63b3149a1bdedc53208a4bf57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17bfeab63b3149a1bdedc53208a4bf57, State: Initialized, Role: FOLLOWER
I20260926 06:20:00.170382 7151 consensus_queue.cc:260] T f74cdeb57897458da52df52465769a23 P 17bfeab63b3149a1bdedc53208a4bf57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:00.170470 7151 ts_tablet_manager.cc:1428] T f74cdeb57897458da52df52465769a23 P 17bfeab63b3149a1bdedc53208a4bf57: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:00.170513 7155 tablet_replica.cc:439] T f74cdeb57897458da52df52465769a23 P 17bfeab63b3149a1bdedc53208a4bf57: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } } }
I20260926 06:20:00.170636 7155 tablet_replica.cc:442] T f74cdeb57897458da52df52465769a23 P 17bfeab63b3149a1bdedc53208a4bf57: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:20:00.194530 7155 raft_consensus.cc:482] T f74cdeb57897458da52df52465769a23 P 17bfeab63b3149a1bdedc53208a4bf57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:20:00.194689 7155 raft_consensus.cc:504] T f74cdeb57897458da52df52465769a23 P 17bfeab63b3149a1bdedc53208a4bf57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:00.194834 7155 leader_election.cc:288] T f74cdeb57897458da52df52465769a23 P 17bfeab63b3149a1bdedc53208a4bf57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233d6dedde314417b8658dadb05c360b (127.4.202.129:33965), 0bab59c2bae246d3a307d3d8af888422 (127.4.202.131:39917)
I20260926 06:20:00.195035 7082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f74cdeb57897458da52df52465769a23" candidate_uuid: "17bfeab63b3149a1bdedc53208a4bf57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bab59c2bae246d3a307d3d8af888422" is_pre_election: true
I20260926 06:20:00.195031 6802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f74cdeb57897458da52df52465769a23" candidate_uuid: "17bfeab63b3149a1bdedc53208a4bf57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233d6dedde314417b8658dadb05c360b" is_pre_election: true
I20260926 06:20:00.195111 7082 raft_consensus.cc:2454] T f74cdeb57897458da52df52465769a23 P 0bab59c2bae246d3a307d3d8af888422 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17bfeab63b3149a1bdedc53208a4bf57 in term 0.
I20260926 06:20:00.195134 6802 raft_consensus.cc:2454] T f74cdeb57897458da52df52465769a23 P 233d6dedde314417b8658dadb05c360b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17bfeab63b3149a1bdedc53208a4bf57 in term 0.
I20260926 06:20:00.195256 6876 leader_election.cc:302] T f74cdeb57897458da52df52465769a23 P 17bfeab63b3149a1bdedc53208a4bf57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bab59c2bae246d3a307d3d8af888422, 17bfeab63b3149a1bdedc53208a4bf57; no voters:
I20260926 06:20:00.195384 7155 raft_consensus.cc:2781] T f74cdeb57897458da52df52465769a23 P 17bfeab63b3149a1bdedc53208a4bf57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:20:00.195433 7155 raft_consensus.cc:482] T f74cdeb57897458da52df52465769a23 P 17bfeab63b3149a1bdedc53208a4bf57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:20:00.195461 7155 raft_consensus.cc:3037] T f74cdeb57897458da52df52465769a23 P 17bfeab63b3149a1bdedc53208a4bf57 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:00.196139 7155 raft_consensus.cc:504] T f74cdeb57897458da52df52465769a23 P 17bfeab63b3149a1bdedc53208a4bf57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:00.196280 7155 leader_election.cc:288] T f74cdeb57897458da52df52465769a23 P 17bfeab63b3149a1bdedc53208a4bf57 [CANDIDATE]: Term 1 election: Requested vote from peers 233d6dedde314417b8658dadb05c360b (127.4.202.129:33965), 0bab59c2bae246d3a307d3d8af888422 (127.4.202.131:39917)
I20260926 06:20:00.196417 7082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f74cdeb57897458da52df52465769a23" candidate_uuid: "17bfeab63b3149a1bdedc53208a4bf57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bab59c2bae246d3a307d3d8af888422"
I20260926 06:20:00.196436 6802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f74cdeb57897458da52df52465769a23" candidate_uuid: "17bfeab63b3149a1bdedc53208a4bf57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233d6dedde314417b8658dadb05c360b"
I20260926 06:20:00.196493 7082 raft_consensus.cc:3037] T f74cdeb57897458da52df52465769a23 P 0bab59c2bae246d3a307d3d8af888422 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:00.196514 6802 raft_consensus.cc:3037] T f74cdeb57897458da52df52465769a23 P 233d6dedde314417b8658dadb05c360b [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:00.197058 7082 raft_consensus.cc:2454] T f74cdeb57897458da52df52465769a23 P 0bab59c2bae246d3a307d3d8af888422 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17bfeab63b3149a1bdedc53208a4bf57 in term 1.
I20260926 06:20:00.197165 6802 raft_consensus.cc:2454] T f74cdeb57897458da52df52465769a23 P 233d6dedde314417b8658dadb05c360b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17bfeab63b3149a1bdedc53208a4bf57 in term 1.
I20260926 06:20:00.197242 6876 leader_election.cc:302] T f74cdeb57897458da52df52465769a23 P 17bfeab63b3149a1bdedc53208a4bf57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bab59c2bae246d3a307d3d8af888422, 17bfeab63b3149a1bdedc53208a4bf57; no voters:
I20260926 06:20:00.197321 7155 raft_consensus.cc:2781] T f74cdeb57897458da52df52465769a23 P 17bfeab63b3149a1bdedc53208a4bf57 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:20:00.197376 7155 raft_consensus.cc:686] T f74cdeb57897458da52df52465769a23 P 17bfeab63b3149a1bdedc53208a4bf57 [term 1 LEADER]: Becoming Leader. State: Replica: 17bfeab63b3149a1bdedc53208a4bf57, State: Running, Role: LEADER
I20260926 06:20:00.197414 7155 consensus_queue.cc:237] T f74cdeb57897458da52df52465769a23 P 17bfeab63b3149a1bdedc53208a4bf57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:00.197662 7161 tablet_replica.cc:439] T f74cdeb57897458da52df52465769a23 P 17bfeab63b3149a1bdedc53208a4bf57: TxnStatusTablet state changed. Reason: New leader 17bfeab63b3149a1bdedc53208a4bf57. Latest consensus state: current_term: 1 leader_uuid: "17bfeab63b3149a1bdedc53208a4bf57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } } }
I20260926 06:20:00.197732 7161 tablet_replica.cc:442] T f74cdeb57897458da52df52465769a23 P 17bfeab63b3149a1bdedc53208a4bf57: This TxnStatusTablet replica's current role is: LEADER
I20260926 06:20:00.197896 6645 catalog_manager.cc:5495] T f74cdeb57897458da52df52465769a23 P 17bfeab63b3149a1bdedc53208a4bf57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 17bfeab63b3149a1bdedc53208a4bf57 (127.4.202.130). New cstate: current_term: 1 leader_uuid: "17bfeab63b3149a1bdedc53208a4bf57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } health_report { overall_health: UNKNOWN } } }
I20260926 06:20:00.198172 7173 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 06:20:00.198292 7173 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 06:20:00.209221 7168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:00.206239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:43075 (local address 127.0.0.1:42464)
0926 06:20:00.206315 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:00.206328 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:00.206377 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:00.206575 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:00.206576 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:00.206625 (+ 49us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 06:20:00.206675 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:00.206677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.207578 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.207579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:00.208201 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:00.208203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.208700 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.208701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:00.208760 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:00.208774 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 06:20:00.209063 (+ 289us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 06:20:00.209069 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 06:20:00.209106 (+ 37us) client_negotiation.cc:240] Negotiation successful
0926 06:20:00.209166 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260926 06:20:00.209225 7148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:00.206337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42464 (local address 127.4.202.130:43075)
0926 06:20:00.206392 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:00.206395 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:00.206406 (+ 11us) server_negotiation.cc:407] Connection header received
0926 06:20:00.206453 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:00.206453 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:00.206471 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:00.206536 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:20:00.206758 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.207486 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.208283 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.208602 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.208672 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:00.208867 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:20:00.208980 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:20:00.209071 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:00.209110 (+ 39us) server_negotiation.cc:299] Negotiation successful
0926 06:20:00.209131 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 06:20:00.210969 6802 raft_consensus.cc:1261] T f74cdeb57897458da52df52465769a23 P 233d6dedde314417b8658dadb05c360b [term 1 FOLLOWER]: Refusing update from remote peer 17bfeab63b3149a1bdedc53208a4bf57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:20:00.210980 7082 raft_consensus.cc:1261] T f74cdeb57897458da52df52465769a23 P 0bab59c2bae246d3a307d3d8af888422 [term 1 FOLLOWER]: Refusing update from remote peer 17bfeab63b3149a1bdedc53208a4bf57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:20:00.211279 7155 consensus_queue.cc:1035] T f74cdeb57897458da52df52465769a23 P 17bfeab63b3149a1bdedc53208a4bf57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:20:00.211457 7155 consensus_queue.cc:1035] T f74cdeb57897458da52df52465769a23 P 17bfeab63b3149a1bdedc53208a4bf57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:20:00.212627 7158 tablet_replica.cc:439] T f74cdeb57897458da52df52465769a23 P 233d6dedde314417b8658dadb05c360b: TxnStatusTablet state changed. Reason: New leader 17bfeab63b3149a1bdedc53208a4bf57. Latest consensus state: current_term: 1 leader_uuid: "17bfeab63b3149a1bdedc53208a4bf57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } } }
I20260926 06:20:00.212634 7159 tablet_replica.cc:439] T f74cdeb57897458da52df52465769a23 P 0bab59c2bae246d3a307d3d8af888422: TxnStatusTablet state changed. Reason: New leader 17bfeab63b3149a1bdedc53208a4bf57. Latest consensus state: current_term: 1 leader_uuid: "17bfeab63b3149a1bdedc53208a4bf57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } } }
I20260926 06:20:00.212733 7158 tablet_replica.cc:442] T f74cdeb57897458da52df52465769a23 P 233d6dedde314417b8658dadb05c360b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:20:00.212738 7159 tablet_replica.cc:442] T f74cdeb57897458da52df52465769a23 P 0bab59c2bae246d3a307d3d8af888422: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:20:00.213806 7161 tablet_replica.cc:439] T f74cdeb57897458da52df52465769a23 P 17bfeab63b3149a1bdedc53208a4bf57: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "17bfeab63b3149a1bdedc53208a4bf57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } } }
I20260926 06:20:00.213883 7161 tablet_replica.cc:442] T f74cdeb57897458da52df52465769a23 P 17bfeab63b3149a1bdedc53208a4bf57: This TxnStatusTablet replica's current role is: LEADER
I20260926 06:20:00.213924 7161 tablet_replica.cc:439] T f74cdeb57897458da52df52465769a23 P 17bfeab63b3149a1bdedc53208a4bf57: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "17bfeab63b3149a1bdedc53208a4bf57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } } }
I20260926 06:20:00.213968 7161 tablet_replica.cc:442] T f74cdeb57897458da52df52465769a23 P 17bfeab63b3149a1bdedc53208a4bf57: This TxnStatusTablet replica's current role is: LEADER
I20260926 06:20:00.214020 7176 mvcc.cc:205] Tried to move back new op lower bound from 7333493146461495296 to 7333493146408898560. Current Snapshot: MvccSnapshot[applied={T|T < 7333493146461495296}]
I20260926 06:20:00.214066 7178 mvcc.cc:205] Tried to move back new op lower bound from 7333493146461495296 to 7333493146408898560. Current Snapshot: MvccSnapshot[applied={T|T < 7333493146461495296}]
I20260926 06:20:00.214172 7159 tablet_replica.cc:439] T f74cdeb57897458da52df52465769a23 P 0bab59c2bae246d3a307d3d8af888422: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "17bfeab63b3149a1bdedc53208a4bf57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } } }
I20260926 06:20:00.214249 7159 tablet_replica.cc:442] T f74cdeb57897458da52df52465769a23 P 0bab59c2bae246d3a307d3d8af888422: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:20:00.214334 7158 tablet_replica.cc:439] T f74cdeb57897458da52df52465769a23 P 233d6dedde314417b8658dadb05c360b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "17bfeab63b3149a1bdedc53208a4bf57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } } }
I20260926 06:20:00.214416 7158 tablet_replica.cc:442] T f74cdeb57897458da52df52465769a23 P 233d6dedde314417b8658dadb05c360b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:20:00.214746 7177 mvcc.cc:205] Tried to move back new op lower bound from 7333493146461495296 to 7333493146408898560. Current Snapshot: MvccSnapshot[applied={T|T < 7333493146461495296 or (T in {7333493146461495296})}]
I20260926 06:20:00.219556 7148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:00.217109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42470 (local address 127.4.202.130:43075)
0926 06:20:00.217160 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:00.217162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:00.217172 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:20:00.217229 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:00.217229 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:00.217246 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:00.217289 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:20:00.217400 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.217847 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.218953 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.219142 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.219201 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:00.219351 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:20:00.219373 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:20:00.219408 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:00.219489 (+ 81us) server_negotiation.cc:299] Negotiation successful
0926 06:20:00.219503 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 06:20:00.225147 7148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:00.222085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:49911 (local address 127.4.202.130:43075)
0926 06:20:00.222127 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:00.222129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:00.222207 (+ 78us) server_negotiation.cc:407] Connection header received
0926 06:20:00.222275 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:00.222276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:00.222285 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:00.222342 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:20:00.222495 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.223057 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.224408 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.224621 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.224702 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:00.224880 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:20:00.224906 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:20:00.224972 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:00.225036 (+ 64us) server_negotiation.cc:299] Negotiation successful
0926 06:20:00.225059 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":24}
I20260926 06:20:00.225207 7188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:00.222004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:43075 (local address 127.4.202.130:49911)
0926 06:20:00.222145 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:00.222155 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:00.222215 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:00.222346 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:00.222347 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:00.222348 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 06:20:00.222400 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:00.222401 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.223165 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.223166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:00.224310 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:00.224312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.224703 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.224704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:00.224772 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:00.224783 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 06:20:00.224972 (+ 189us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 06:20:00.224974 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:20:00.225032 (+ 58us) client_negotiation.cc:240] Negotiation successful
0926 06:20:00.225076 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":1,"thread_start_us":80,"threads_started":1}
I20260926 06:20:00.227185 6802 raft_consensus.cc:1261] T d0c11e6cd4e6478d8df8b54e58315e4f P 233d6dedde314417b8658dadb05c360b [term 1 FOLLOWER]: Refusing update from remote peer 17bfeab63b3149a1bdedc53208a4bf57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:20:00.227182 7082 raft_consensus.cc:1261] T d0c11e6cd4e6478d8df8b54e58315e4f P 0bab59c2bae246d3a307d3d8af888422 [term 1 FOLLOWER]: Refusing update from remote peer 17bfeab63b3149a1bdedc53208a4bf57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:20:00.227370 7155 consensus_queue.cc:1035] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:20:00.227456 7181 consensus_queue.cc:1035] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:20:00.238808 6645 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:20:00.238921 6645 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:20:00.239665 6645 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4707268ef11e42139ecf487c90adb27a: Sending DeleteTablet for 3 replicas of tablet d0c11e6cd4e6478d8df8b54e58315e4f
I20260926 06:20:00.240002 7062 tablet_service.cc:1514] Processing DeleteTablet for tablet d0c11e6cd4e6478d8df8b54e58315e4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:20:00 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:54084
I20260926 06:20:00.240002 6782 tablet_service.cc:1514] Processing DeleteTablet for tablet d0c11e6cd4e6478d8df8b54e58315e4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:20:00 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:58658
I20260926 06:20:00.240199 6922 tablet_service.cc:1514] Processing DeleteTablet for tablet d0c11e6cd4e6478d8df8b54e58315e4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:20:00 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:42460
I20260926 06:20:00.240409 7194 tablet_replica.cc:330] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57: stopping tablet replica
I20260926 06:20:00.240391 6645 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36530:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:20:00.240504 6645 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:20:00.240537 7194 raft_consensus.cc:2229] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57 [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:20:00.240630 7194 raft_consensus.cc:2258] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:20:00.240870 7192 tablet_replica.cc:330] T d0c11e6cd4e6478d8df8b54e58315e4f P 0bab59c2bae246d3a307d3d8af888422: stopping tablet replica
I20260926 06:20:00.240960 7192 raft_consensus.cc:2229] T d0c11e6cd4e6478d8df8b54e58315e4f P 0bab59c2bae246d3a307d3d8af888422 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:20:00.241004 7192 raft_consensus.cc:2258] T d0c11e6cd4e6478d8df8b54e58315e4f P 0bab59c2bae246d3a307d3d8af888422 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:20:00.241384 7193 tablet_replica.cc:330] T d0c11e6cd4e6478d8df8b54e58315e4f P 233d6dedde314417b8658dadb05c360b: stopping tablet replica
I20260926 06:20:00.241465 7193 raft_consensus.cc:2229] T d0c11e6cd4e6478d8df8b54e58315e4f P 233d6dedde314417b8658dadb05c360b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:20:00.241796 6922 tablet_service.cc:1467] Processing CreateTablet for tablet 6ecc1c68db0f411390f07b36f7981ebb (DEFAULT_TABLE table=test-table [id=56522911b202461890ac2e7fa9930ef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:20:00.241879 7194 ts_tablet_manager.cc:1905] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:20:00.241918 6922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ecc1c68db0f411390f07b36f7981ebb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:00.241905 7062 tablet_service.cc:1467] Processing CreateTablet for tablet 6ecc1c68db0f411390f07b36f7981ebb (DEFAULT_TABLE table=test-table [id=56522911b202461890ac2e7fa9930ef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:20:00.242008 7062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ecc1c68db0f411390f07b36f7981ebb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:00.243325 7151 tablet_bootstrap.cc:492] T 6ecc1c68db0f411390f07b36f7981ebb P 17bfeab63b3149a1bdedc53208a4bf57: Bootstrap starting.
I20260926 06:20:00.243512 7154 tablet_bootstrap.cc:492] T 6ecc1c68db0f411390f07b36f7981ebb P 0bab59c2bae246d3a307d3d8af888422: Bootstrap starting.
I20260926 06:20:00.243732 7151 tablet_bootstrap.cc:654] T 6ecc1c68db0f411390f07b36f7981ebb P 17bfeab63b3149a1bdedc53208a4bf57: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:00.243875 7154 tablet_bootstrap.cc:654] T 6ecc1c68db0f411390f07b36f7981ebb P 0bab59c2bae246d3a307d3d8af888422: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:00.244300 7151 tablet_bootstrap.cc:492] T 6ecc1c68db0f411390f07b36f7981ebb P 17bfeab63b3149a1bdedc53208a4bf57: No bootstrap required, opened a new log
I20260926 06:20:00.244342 7151 ts_tablet_manager.cc:1397] T 6ecc1c68db0f411390f07b36f7981ebb P 17bfeab63b3149a1bdedc53208a4bf57: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 06:20:00.244477 7151 raft_consensus.cc:348] T 6ecc1c68db0f411390f07b36f7981ebb P 17bfeab63b3149a1bdedc53208a4bf57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:00.244531 7151 raft_consensus.cc:374] T 6ecc1c68db0f411390f07b36f7981ebb P 17bfeab63b3149a1bdedc53208a4bf57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:00.244561 7151 raft_consensus.cc:729] T 6ecc1c68db0f411390f07b36f7981ebb P 17bfeab63b3149a1bdedc53208a4bf57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17bfeab63b3149a1bdedc53208a4bf57, State: Initialized, Role: FOLLOWER
I20260926 06:20:00.244598 7151 consensus_queue.cc:260] T 6ecc1c68db0f411390f07b36f7981ebb P 17bfeab63b3149a1bdedc53208a4bf57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:00.244596 7194 ts_tablet_manager.cc:1918] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 06:20:00.244685 7151 ts_tablet_manager.cc:1428] T 6ecc1c68db0f411390f07b36f7981ebb P 17bfeab63b3149a1bdedc53208a4bf57: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:00.244707 7194 log.cc:1196] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-1/wal/wals/d0c11e6cd4e6478d8df8b54e58315e4f
I20260926 06:20:00.245107 7192 ts_tablet_manager.cc:1905] T d0c11e6cd4e6478d8df8b54e58315e4f P 0bab59c2bae246d3a307d3d8af888422: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:20:00.245286 7194 ts_tablet_manager.cc:1939] T d0c11e6cd4e6478d8df8b54e58315e4f P 17bfeab63b3149a1bdedc53208a4bf57: Deleting consensus metadata
I20260926 06:20:00.245663 6631 catalog_manager.cc:4841] TS 17bfeab63b3149a1bdedc53208a4bf57 (127.4.202.130:43075): tablet d0c11e6cd4e6478d8df8b54e58315e4f (table test-table [id=e40eeceffd3c49b1a3de63d5c30ddd22]) successfully deleted
I20260926 06:20:00.246516 7192 ts_tablet_manager.cc:1918] T d0c11e6cd4e6478d8df8b54e58315e4f P 0bab59c2bae246d3a307d3d8af888422: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 06:20:00.246614 7192 log.cc:1196] T d0c11e6cd4e6478d8df8b54e58315e4f P 0bab59c2bae246d3a307d3d8af888422: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-2/wal/wals/d0c11e6cd4e6478d8df8b54e58315e4f
I20260926 06:20:00.246840 7192 ts_tablet_manager.cc:1939] T d0c11e6cd4e6478d8df8b54e58315e4f P 0bab59c2bae246d3a307d3d8af888422: Deleting consensus metadata
I20260926 06:20:00.241541 7193 raft_consensus.cc:2258] T d0c11e6cd4e6478d8df8b54e58315e4f P 233d6dedde314417b8658dadb05c360b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:20:00.241784 6782 tablet_service.cc:1467] Processing CreateTablet for tablet 6ecc1c68db0f411390f07b36f7981ebb (DEFAULT_TABLE table=test-table [id=56522911b202461890ac2e7fa9930ef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:20:00.247249 6632 catalog_manager.cc:4841] TS 0bab59c2bae246d3a307d3d8af888422 (127.4.202.131:39917): tablet d0c11e6cd4e6478d8df8b54e58315e4f (table test-table [id=e40eeceffd3c49b1a3de63d5c30ddd22]) successfully deleted
I20260926 06:20:00.247292 6782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ecc1c68db0f411390f07b36f7981ebb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:00.248360 7152 tablet_bootstrap.cc:492] T 6ecc1c68db0f411390f07b36f7981ebb P 233d6dedde314417b8658dadb05c360b: Bootstrap starting.
I20260926 06:20:00.248710 7193 ts_tablet_manager.cc:1905] T d0c11e6cd4e6478d8df8b54e58315e4f P 233d6dedde314417b8658dadb05c360b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:20:00.248819 7152 tablet_bootstrap.cc:654] T 6ecc1c68db0f411390f07b36f7981ebb P 233d6dedde314417b8658dadb05c360b: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:00.249503 7152 tablet_bootstrap.cc:492] T 6ecc1c68db0f411390f07b36f7981ebb P 233d6dedde314417b8658dadb05c360b: No bootstrap required, opened a new log
I20260926 06:20:00.249559 7152 ts_tablet_manager.cc:1397] T 6ecc1c68db0f411390f07b36f7981ebb P 233d6dedde314417b8658dadb05c360b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 06:20:00.249682 7152 raft_consensus.cc:348] T 6ecc1c68db0f411390f07b36f7981ebb P 233d6dedde314417b8658dadb05c360b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:00.249727 7152 raft_consensus.cc:374] T 6ecc1c68db0f411390f07b36f7981ebb P 233d6dedde314417b8658dadb05c360b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:00.249742 7152 raft_consensus.cc:729] T 6ecc1c68db0f411390f07b36f7981ebb P 233d6dedde314417b8658dadb05c360b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233d6dedde314417b8658dadb05c360b, State: Initialized, Role: FOLLOWER
I20260926 06:20:00.249801 7152 consensus_queue.cc:260] T 6ecc1c68db0f411390f07b36f7981ebb P 233d6dedde314417b8658dadb05c360b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:00.249895 7152 ts_tablet_manager.cc:1428] T 6ecc1c68db0f411390f07b36f7981ebb P 233d6dedde314417b8658dadb05c360b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:00.250180 7193 ts_tablet_manager.cc:1918] T d0c11e6cd4e6478d8df8b54e58315e4f P 233d6dedde314417b8658dadb05c360b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 06:20:00.250247 7193 log.cc:1196] T d0c11e6cd4e6478d8df8b54e58315e4f P 233d6dedde314417b8658dadb05c360b: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-0/wal/wals/d0c11e6cd4e6478d8df8b54e58315e4f
I20260926 06:20:00.250617 7193 ts_tablet_manager.cc:1939] T d0c11e6cd4e6478d8df8b54e58315e4f P 233d6dedde314417b8658dadb05c360b: Deleting consensus metadata
I20260926 06:20:00.250901 7154 tablet_bootstrap.cc:492] T 6ecc1c68db0f411390f07b36f7981ebb P 0bab59c2bae246d3a307d3d8af888422: No bootstrap required, opened a new log
I20260926 06:20:00.250976 7154 ts_tablet_manager.cc:1397] T 6ecc1c68db0f411390f07b36f7981ebb P 0bab59c2bae246d3a307d3d8af888422: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260926 06:20:00.251020 6630 catalog_manager.cc:4841] TS 233d6dedde314417b8658dadb05c360b (127.4.202.129:33965): tablet d0c11e6cd4e6478d8df8b54e58315e4f (table test-table [id=e40eeceffd3c49b1a3de63d5c30ddd22]) successfully deleted
I20260926 06:20:00.251137 7154 raft_consensus.cc:348] T 6ecc1c68db0f411390f07b36f7981ebb P 0bab59c2bae246d3a307d3d8af888422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:00.251183 7154 raft_consensus.cc:374] T 6ecc1c68db0f411390f07b36f7981ebb P 0bab59c2bae246d3a307d3d8af888422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:00.251196 7154 raft_consensus.cc:729] T 6ecc1c68db0f411390f07b36f7981ebb P 0bab59c2bae246d3a307d3d8af888422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bab59c2bae246d3a307d3d8af888422, State: Initialized, Role: FOLLOWER
I20260926 06:20:00.251231 7154 consensus_queue.cc:260] T 6ecc1c68db0f411390f07b36f7981ebb P 0bab59c2bae246d3a307d3d8af888422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:00.251343 7154 ts_tablet_manager.cc:1428] T 6ecc1c68db0f411390f07b36f7981ebb P 0bab59c2bae246d3a307d3d8af888422: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:00.470651 7158 raft_consensus.cc:482] T 6ecc1c68db0f411390f07b36f7981ebb P 233d6dedde314417b8658dadb05c360b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:20:00.470734 7158 raft_consensus.cc:504] T 6ecc1c68db0f411390f07b36f7981ebb P 233d6dedde314417b8658dadb05c360b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:00.471100 7158 leader_election.cc:288] T 6ecc1c68db0f411390f07b36f7981ebb P 233d6dedde314417b8658dadb05c360b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17bfeab63b3149a1bdedc53208a4bf57 (127.4.202.130:43075), 0bab59c2bae246d3a307d3d8af888422 (127.4.202.131:39917)
I20260926 06:20:00.474520 7148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:00.471387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:38187 (local address 127.4.202.130:43075)
0926 06:20:00.471427 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:00.471429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:00.471455 (+ 26us) server_negotiation.cc:407] Connection header received
0926 06:20:00.471537 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:00.471538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:00.471547 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:00.471602 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:20:00.471819 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.472588 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.473712 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.474143 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.474231 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:00.474259 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:00.474406 (+ 147us) server_negotiation.cc:299] Negotiation successful
0926 06:20:00.474456 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 06:20:00.474524 7199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:00.471268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:43075 (local address 127.4.202.129:38187)
0926 06:20:00.471409 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:00.471418 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:00.471470 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:00.471619 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:00.471620 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:00.471623 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:20:00.471737 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:00.471739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.472703 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.472704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:00.473641 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:00.473644 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.474239 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.474258 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:00.474329 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:00.474333 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:20:00.474387 (+ 54us) client_negotiation.cc:240] Negotiation successful
0926 06:20:00.474435 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":43,"thread_start_us":59,"threads_started":1}
I20260926 06:20:00.475153 7198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:00.471267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:39917 (local address 127.4.202.129:57453)
0926 06:20:00.471391 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:00.471404 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:00.471470 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:00.471619 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:00.471620 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:00.471622 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:20:00.471739 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:00.471739 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.472688 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.472689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:00.473951 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:00.473954 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.474880 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.474881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:00.474959 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:00.474962 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:20:00.475039 (+ 77us) client_negotiation.cc:240] Negotiation successful
0926 06:20:00.475075 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":65,"thread_start_us":78,"threads_started":1}
I20260926 06:20:00.475153 7150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:00.471322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:57453 (local address 127.4.202.131:39917)
0926 06:20:00.471371 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:00.471373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:00.471456 (+ 83us) server_negotiation.cc:407] Connection header received
0926 06:20:00.471539 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:00.471539 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:00.471547 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:00.471602 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:20:00.471820 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.472603 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.474153 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.474798 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.474877 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:00.474885 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:00.475049 (+ 164us) server_negotiation.cc:299] Negotiation successful
0926 06:20:00.475089 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 06:20:00.475277 6942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ecc1c68db0f411390f07b36f7981ebb" candidate_uuid: "233d6dedde314417b8658dadb05c360b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17bfeab63b3149a1bdedc53208a4bf57" is_pre_election: true
I20260926 06:20:00.475425 6942 raft_consensus.cc:2454] T 6ecc1c68db0f411390f07b36f7981ebb P 17bfeab63b3149a1bdedc53208a4bf57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 233d6dedde314417b8658dadb05c360b in term 0.
I20260926 06:20:00.475638 7082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ecc1c68db0f411390f07b36f7981ebb" candidate_uuid: "233d6dedde314417b8658dadb05c360b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bab59c2bae246d3a307d3d8af888422" is_pre_election: true
I20260926 06:20:00.475733 7082 raft_consensus.cc:2454] T 6ecc1c68db0f411390f07b36f7981ebb P 0bab59c2bae246d3a307d3d8af888422 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 233d6dedde314417b8658dadb05c360b in term 0.
I20260926 06:20:00.475746 6735 leader_election.cc:302] T 6ecc1c68db0f411390f07b36f7981ebb P 233d6dedde314417b8658dadb05c360b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17bfeab63b3149a1bdedc53208a4bf57, 233d6dedde314417b8658dadb05c360b; no voters:
I20260926 06:20:00.475859 7158 raft_consensus.cc:2781] T 6ecc1c68db0f411390f07b36f7981ebb P 233d6dedde314417b8658dadb05c360b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:20:00.475901 7158 raft_consensus.cc:482] T 6ecc1c68db0f411390f07b36f7981ebb P 233d6dedde314417b8658dadb05c360b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:20:00.475922 7158 raft_consensus.cc:3037] T 6ecc1c68db0f411390f07b36f7981ebb P 233d6dedde314417b8658dadb05c360b [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:00.476711 7158 raft_consensus.cc:504] T 6ecc1c68db0f411390f07b36f7981ebb P 233d6dedde314417b8658dadb05c360b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:00.476842 7158 leader_election.cc:288] T 6ecc1c68db0f411390f07b36f7981ebb P 233d6dedde314417b8658dadb05c360b [CANDIDATE]: Term 1 election: Requested vote from peers 17bfeab63b3149a1bdedc53208a4bf57 (127.4.202.130:43075), 0bab59c2bae246d3a307d3d8af888422 (127.4.202.131:39917)
I20260926 06:20:00.476995 6942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ecc1c68db0f411390f07b36f7981ebb" candidate_uuid: "233d6dedde314417b8658dadb05c360b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17bfeab63b3149a1bdedc53208a4bf57"
I20260926 06:20:00.477003 7082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ecc1c68db0f411390f07b36f7981ebb" candidate_uuid: "233d6dedde314417b8658dadb05c360b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bab59c2bae246d3a307d3d8af888422"
I20260926 06:20:00.477080 6942 raft_consensus.cc:3037] T 6ecc1c68db0f411390f07b36f7981ebb P 17bfeab63b3149a1bdedc53208a4bf57 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:00.477084 7082 raft_consensus.cc:3037] T 6ecc1c68db0f411390f07b36f7981ebb P 0bab59c2bae246d3a307d3d8af888422 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:00.477833 7082 raft_consensus.cc:2454] T 6ecc1c68db0f411390f07b36f7981ebb P 0bab59c2bae246d3a307d3d8af888422 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 233d6dedde314417b8658dadb05c360b in term 1.
I20260926 06:20:00.477958 6942 raft_consensus.cc:2454] T 6ecc1c68db0f411390f07b36f7981ebb P 17bfeab63b3149a1bdedc53208a4bf57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 233d6dedde314417b8658dadb05c360b in term 1.
I20260926 06:20:00.478010 6736 leader_election.cc:302] T 6ecc1c68db0f411390f07b36f7981ebb P 233d6dedde314417b8658dadb05c360b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bab59c2bae246d3a307d3d8af888422, 233d6dedde314417b8658dadb05c360b; no voters:
I20260926 06:20:00.478089 7158 raft_consensus.cc:2781] T 6ecc1c68db0f411390f07b36f7981ebb P 233d6dedde314417b8658dadb05c360b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:20:00.478228 7158 raft_consensus.cc:686] T 6ecc1c68db0f411390f07b36f7981ebb P 233d6dedde314417b8658dadb05c360b [term 1 LEADER]: Becoming Leader. State: Replica: 233d6dedde314417b8658dadb05c360b, State: Running, Role: LEADER
I20260926 06:20:00.478322 7158 consensus_queue.cc:237] T 6ecc1c68db0f411390f07b36f7981ebb P 233d6dedde314417b8658dadb05c360b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:00.479009 6645 catalog_manager.cc:5495] T 6ecc1c68db0f411390f07b36f7981ebb P 233d6dedde314417b8658dadb05c360b reported cstate change: term changed from 0 to 1, leader changed from <none> to 233d6dedde314417b8658dadb05c360b (127.4.202.129). New cstate: current_term: 1 leader_uuid: "233d6dedde314417b8658dadb05c360b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } health_report { overall_health: UNKNOWN } } }
I20260926 06:20:00.530772 7149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:00.528271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58662 (local address 127.4.202.129:33965)
0926 06:20:00.528315 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:00.528318 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:00.528327 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:20:00.528369 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:00.528370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:00.528390 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:00.528485 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:20:00.528625 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.529201 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.529939 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.530145 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.530250 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:00.530430 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:20:00.530545 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:20:00.530651 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:00.530674 (+ 23us) server_negotiation.cc:299] Negotiation successful
0926 06:20:00.530692 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 06:20:00.535537 7148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:00.533470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:60925 (local address 127.4.202.130:43075)
0926 06:20:00.533512 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:00.533514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:00.533611 (+ 97us) server_negotiation.cc:407] Connection header received
0926 06:20:00.533646 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:00.533647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:00.533655 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:00.533688 (+ 33us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:20:00.533829 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.534294 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.534992 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.535168 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.535217 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:00.535367 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:20:00.535391 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:20:00.535426 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:00.535476 (+ 50us) server_negotiation.cc:299] Negotiation successful
0926 06:20:00.535489 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 06:20:00.535625 7204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:00.533389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:43075 (local address 127.4.202.129:60925)
0926 06:20:00.533542 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:00.533551 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:00.533601 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:00.533700 (+ 99us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:00.533701 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:00.533703 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 06:20:00.533773 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:00.533774 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.534350 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.534350 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:00.534930 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:00.534932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.535245 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.535246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:00.535292 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:00.535304 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 06:20:00.535441 (+ 137us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 06:20:00.535442 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 06:20:00.535476 (+ 34us) client_negotiation.cc:240] Negotiation successful
0926 06:20:00.535509 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":80,"threads_started":1}
I20260926 06:20:00.537557 7082 raft_consensus.cc:1261] T 6ecc1c68db0f411390f07b36f7981ebb P 0bab59c2bae246d3a307d3d8af888422 [term 1 FOLLOWER]: Refusing update from remote peer 233d6dedde314417b8658dadb05c360b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:20:00.537649 6942 raft_consensus.cc:1261] T 6ecc1c68db0f411390f07b36f7981ebb P 17bfeab63b3149a1bdedc53208a4bf57 [term 1 FOLLOWER]: Refusing update from remote peer 233d6dedde314417b8658dadb05c360b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:20:00.537860 7200 consensus_queue.cc:1035] T 6ecc1c68db0f411390f07b36f7981ebb P 233d6dedde314417b8658dadb05c360b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:20:00.538017 7158 consensus_queue.cc:1035] T 6ecc1c68db0f411390f07b36f7981ebb P 233d6dedde314417b8658dadb05c360b [LEADER]: Connected to new peer: Peer: permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:20:00.544294 7188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:00.542056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:33965 (local address 127.4.202.130:60103)
0926 06:20:00.542112 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:00.542123 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:00.542162 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:00.542297 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:00.542297 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:00.542298 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 06:20:00.542335 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:00.542336 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.543108 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.543108 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:00.543648 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:00.543650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.543935 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.543936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:00.543976 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:00.543980 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 06:20:00.544128 (+ 148us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 06:20:00.544129 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 06:20:00.544179 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 06:20:00.544222 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260926 06:20:00.544301 7149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:00.542152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:60103 (local address 127.4.202.129:33965)
0926 06:20:00.542205 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:00.542207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:00.542214 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:20:00.542231 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:00.542232 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:00.542240 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:00.542292 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:20:00.542412 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.543028 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.543715 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:00.543873 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:00.543940 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:00.544053 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:20:00.544083 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:20:00.544128 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:00.544198 (+ 70us) server_negotiation.cc:299] Negotiation successful
0926 06:20:00.544233 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260926 06:20:00.544592 7177 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260926 06:20:02.069991 6645 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:20:02.070106 6645 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:20:02.070937 6645 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4707268ef11e42139ecf487c90adb27a: Sending DeleteTablet for 3 replicas of tablet 6ecc1c68db0f411390f07b36f7981ebb
I20260926 06:20:02.071187 6782 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ecc1c68db0f411390f07b36f7981ebb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:20:02 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:58658
I20260926 06:20:02.071204 7062 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ecc1c68db0f411390f07b36f7981ebb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:20:02 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:54084
I20260926 06:20:02.071265 6922 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ecc1c68db0f411390f07b36f7981ebb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:20:02 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:42460
I20260926 06:20:02.071391 7230 tablet_replica.cc:330] T 6ecc1c68db0f411390f07b36f7981ebb P 233d6dedde314417b8658dadb05c360b: stopping tablet replica
I20260926 06:20:02.071491 7230 raft_consensus.cc:2229] T 6ecc1c68db0f411390f07b36f7981ebb P 233d6dedde314417b8658dadb05c360b [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:20:02.071636 7230 raft_consensus.cc:2258] T 6ecc1c68db0f411390f07b36f7981ebb P 233d6dedde314417b8658dadb05c360b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:20:02.071933 7230 ts_tablet_manager.cc:1905] T 6ecc1c68db0f411390f07b36f7981ebb P 233d6dedde314417b8658dadb05c360b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:20:02.073786 7231 tablet_replica.cc:330] T 6ecc1c68db0f411390f07b36f7981ebb P 0bab59c2bae246d3a307d3d8af888422: stopping tablet replica
I20260926 06:20:02.073860 7231 raft_consensus.cc:2229] T 6ecc1c68db0f411390f07b36f7981ebb P 0bab59c2bae246d3a307d3d8af888422 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:20:02.073899 7231 raft_consensus.cc:2258] T 6ecc1c68db0f411390f07b36f7981ebb P 0bab59c2bae246d3a307d3d8af888422 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:20:02.074028 7230 ts_tablet_manager.cc:1918] T 6ecc1c68db0f411390f07b36f7981ebb P 233d6dedde314417b8658dadb05c360b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 06:20:02.074083 7230 log.cc:1196] T 6ecc1c68db0f411390f07b36f7981ebb P 233d6dedde314417b8658dadb05c360b: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-0/wal/wals/6ecc1c68db0f411390f07b36f7981ebb
I20260926 06:20:02.074209 7231 ts_tablet_manager.cc:1905] T 6ecc1c68db0f411390f07b36f7981ebb P 0bab59c2bae246d3a307d3d8af888422: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:20:02.074323 7230 ts_tablet_manager.cc:1939] T 6ecc1c68db0f411390f07b36f7981ebb P 233d6dedde314417b8658dadb05c360b: Deleting consensus metadata
I20260926 06:20:02.074608 7232 tablet_replica.cc:330] T 6ecc1c68db0f411390f07b36f7981ebb P 17bfeab63b3149a1bdedc53208a4bf57: stopping tablet replica
I20260926 06:20:02.074631 6630 catalog_manager.cc:4841] TS 233d6dedde314417b8658dadb05c360b (127.4.202.129:33965): tablet 6ecc1c68db0f411390f07b36f7981ebb (table test-table [id=56522911b202461890ac2e7fa9930ef0]) successfully deleted
I20260926 06:20:02.074667 7232 raft_consensus.cc:2229] T 6ecc1c68db0f411390f07b36f7981ebb P 17bfeab63b3149a1bdedc53208a4bf57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:20:02.074698 7232 raft_consensus.cc:2258] T 6ecc1c68db0f411390f07b36f7981ebb P 17bfeab63b3149a1bdedc53208a4bf57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:20:02.074878 7232 ts_tablet_manager.cc:1905] T 6ecc1c68db0f411390f07b36f7981ebb P 17bfeab63b3149a1bdedc53208a4bf57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:20:02.075732 7231 ts_tablet_manager.cc:1918] T 6ecc1c68db0f411390f07b36f7981ebb P 0bab59c2bae246d3a307d3d8af888422: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 06:20:02.075802 7231 log.cc:1196] T 6ecc1c68db0f411390f07b36f7981ebb P 0bab59c2bae246d3a307d3d8af888422: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-2/wal/wals/6ecc1c68db0f411390f07b36f7981ebb
I20260926 06:20:02.076009 7231 ts_tablet_manager.cc:1939] T 6ecc1c68db0f411390f07b36f7981ebb P 0bab59c2bae246d3a307d3d8af888422: Deleting consensus metadata
I20260926 06:20:02.076452 6632 catalog_manager.cc:4841] TS 0bab59c2bae246d3a307d3d8af888422 (127.4.202.131:39917): tablet 6ecc1c68db0f411390f07b36f7981ebb (table test-table [id=56522911b202461890ac2e7fa9930ef0]) successfully deleted
I20260926 06:20:02.076773 7232 ts_tablet_manager.cc:1918] T 6ecc1c68db0f411390f07b36f7981ebb P 17bfeab63b3149a1bdedc53208a4bf57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 06:20:02.076838 7232 log.cc:1196] T 6ecc1c68db0f411390f07b36f7981ebb P 17bfeab63b3149a1bdedc53208a4bf57: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-1/wal/wals/6ecc1c68db0f411390f07b36f7981ebb
I20260926 06:20:02.077080 7232 ts_tablet_manager.cc:1939] T 6ecc1c68db0f411390f07b36f7981ebb P 17bfeab63b3149a1bdedc53208a4bf57: Deleting consensus metadata
I20260926 06:20:02.077432 6631 catalog_manager.cc:4841] TS 17bfeab63b3149a1bdedc53208a4bf57 (127.4.202.130:43075): tablet 6ecc1c68db0f411390f07b36f7981ebb (table test-table [id=56522911b202461890ac2e7fa9930ef0]) successfully deleted
Sep 26 06:20:02 dist-test-slave-pd0t krb5kdc[6600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403602, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 06:20:02 dist-test-slave-pd0t krb5kdc[6600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403602, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:20:02.096632 7240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:02.089422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36544 (local address 127.4.202.190:38709)
0926 06:20:02.089592 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:02.089594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:02.089604 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:20:02.089693 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:02.089693 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:02.089707 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:02.089768 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:02.090292 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:02.090878 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:02.091623 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:02.091823 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:02.091913 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:02.094115 (+ 2202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:02.094117 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:02.094117 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:02.094124 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:02.095615 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:02.095914 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:02.095914 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:02.095915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:02.095963 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:02.096126 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:02.096126 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:02.096126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:02.096221 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:02.096294 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:02.096413 (+ 119us) server_negotiation.cc:299] Negotiation successful
0926 06:20:02.096474 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":74,"threads_started":1}
I20260926 06:20:02.097946 6645 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36544:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:20:02.098106 6645 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:20:02.099381 6782 tablet_service.cc:1467] Processing CreateTablet for tablet 897704d052154173944c901b5026d1ed (DEFAULT_TABLE table=test-table [id=b5131d90486649f9bb8f03de1f96b593]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:20:02.099417 6922 tablet_service.cc:1467] Processing CreateTablet for tablet 897704d052154173944c901b5026d1ed (DEFAULT_TABLE table=test-table [id=b5131d90486649f9bb8f03de1f96b593]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:20:02.099509 6782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 897704d052154173944c901b5026d1ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:02.099555 6922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 897704d052154173944c901b5026d1ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:02.100548 7241 tablet_bootstrap.cc:492] T 897704d052154173944c901b5026d1ed P 233d6dedde314417b8658dadb05c360b: Bootstrap starting.
I20260926 06:20:02.100648 7062 tablet_service.cc:1467] Processing CreateTablet for tablet 897704d052154173944c901b5026d1ed (DEFAULT_TABLE table=test-table [id=b5131d90486649f9bb8f03de1f96b593]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:20:02.100745 7062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 897704d052154173944c901b5026d1ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:02.101003 7241 tablet_bootstrap.cc:654] T 897704d052154173944c901b5026d1ed P 233d6dedde314417b8658dadb05c360b: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:02.101557 7241 tablet_bootstrap.cc:492] T 897704d052154173944c901b5026d1ed P 233d6dedde314417b8658dadb05c360b: No bootstrap required, opened a new log
I20260926 06:20:02.101612 7241 ts_tablet_manager.cc:1397] T 897704d052154173944c901b5026d1ed P 233d6dedde314417b8658dadb05c360b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 06:20:02.101719 7241 raft_consensus.cc:348] T 897704d052154173944c901b5026d1ed P 233d6dedde314417b8658dadb05c360b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:02.101778 7241 raft_consensus.cc:374] T 897704d052154173944c901b5026d1ed P 233d6dedde314417b8658dadb05c360b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:02.101796 7241 raft_consensus.cc:729] T 897704d052154173944c901b5026d1ed P 233d6dedde314417b8658dadb05c360b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233d6dedde314417b8658dadb05c360b, State: Initialized, Role: FOLLOWER
I20260926 06:20:02.101843 7241 consensus_queue.cc:260] T 897704d052154173944c901b5026d1ed P 233d6dedde314417b8658dadb05c360b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:02.101948 7241 ts_tablet_manager.cc:1428] T 897704d052154173944c901b5026d1ed P 233d6dedde314417b8658dadb05c360b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:02.102130 7244 tablet_bootstrap.cc:492] T 897704d052154173944c901b5026d1ed P 0bab59c2bae246d3a307d3d8af888422: Bootstrap starting.
I20260926 06:20:02.102496 7244 tablet_bootstrap.cc:654] T 897704d052154173944c901b5026d1ed P 0bab59c2bae246d3a307d3d8af888422: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:02.102999 7242 tablet_bootstrap.cc:492] T 897704d052154173944c901b5026d1ed P 17bfeab63b3149a1bdedc53208a4bf57: Bootstrap starting.
I20260926 06:20:02.103327 7242 tablet_bootstrap.cc:654] T 897704d052154173944c901b5026d1ed P 17bfeab63b3149a1bdedc53208a4bf57: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:02.103842 7244 tablet_bootstrap.cc:492] T 897704d052154173944c901b5026d1ed P 0bab59c2bae246d3a307d3d8af888422: No bootstrap required, opened a new log
I20260926 06:20:02.103852 7242 tablet_bootstrap.cc:492] T 897704d052154173944c901b5026d1ed P 17bfeab63b3149a1bdedc53208a4bf57: No bootstrap required, opened a new log
I20260926 06:20:02.103896 7242 ts_tablet_manager.cc:1397] T 897704d052154173944c901b5026d1ed P 17bfeab63b3149a1bdedc53208a4bf57: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 06:20:02.103896 7244 ts_tablet_manager.cc:1397] T 897704d052154173944c901b5026d1ed P 0bab59c2bae246d3a307d3d8af888422: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 06:20:02.103994 7244 raft_consensus.cc:348] T 897704d052154173944c901b5026d1ed P 0bab59c2bae246d3a307d3d8af888422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:02.103994 7242 raft_consensus.cc:348] T 897704d052154173944c901b5026d1ed P 17bfeab63b3149a1bdedc53208a4bf57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:02.104051 7244 raft_consensus.cc:374] T 897704d052154173944c901b5026d1ed P 0bab59c2bae246d3a307d3d8af888422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:02.104053 7242 raft_consensus.cc:374] T 897704d052154173944c901b5026d1ed P 17bfeab63b3149a1bdedc53208a4bf57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:02.104084 7244 raft_consensus.cc:729] T 897704d052154173944c901b5026d1ed P 0bab59c2bae246d3a307d3d8af888422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bab59c2bae246d3a307d3d8af888422, State: Initialized, Role: FOLLOWER
I20260926 06:20:02.104084 7242 raft_consensus.cc:729] T 897704d052154173944c901b5026d1ed P 17bfeab63b3149a1bdedc53208a4bf57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17bfeab63b3149a1bdedc53208a4bf57, State: Initialized, Role: FOLLOWER
I20260926 06:20:02.104135 7244 consensus_queue.cc:260] T 897704d052154173944c901b5026d1ed P 0bab59c2bae246d3a307d3d8af888422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:02.104194 7242 consensus_queue.cc:260] T 897704d052154173944c901b5026d1ed P 17bfeab63b3149a1bdedc53208a4bf57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:02.104315 7244 ts_tablet_manager.cc:1428] T 897704d052154173944c901b5026d1ed P 0bab59c2bae246d3a307d3d8af888422: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260926 06:20:02.104472 7242 ts_tablet_manager.cc:1428] T 897704d052154173944c901b5026d1ed P 17bfeab63b3149a1bdedc53208a4bf57: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 06:20:02.212659 7247 raft_consensus.cc:482] T 897704d052154173944c901b5026d1ed P 0bab59c2bae246d3a307d3d8af888422 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:20:02.212744 7247 raft_consensus.cc:504] T 897704d052154173944c901b5026d1ed P 0bab59c2bae246d3a307d3d8af888422 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:02.213115 7247 leader_election.cc:288] T 897704d052154173944c901b5026d1ed P 0bab59c2bae246d3a307d3d8af888422 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233d6dedde314417b8658dadb05c360b (127.4.202.129:33965), 17bfeab63b3149a1bdedc53208a4bf57 (127.4.202.130:43075)
I20260926 06:20:02.217255 7250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:02.213403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:48063 (local address 127.4.202.130:43075)
0926 06:20:02.213532 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:02.213534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:02.213569 (+ 35us) server_negotiation.cc:407] Connection header received
0926 06:20:02.213691 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:02.213693 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:02.213703 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:02.213786 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:20:02.214317 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:02.214981 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:02.216273 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:02.216825 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:02.216905 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:02.216915 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:02.217135 (+ 220us) server_negotiation.cc:299] Negotiation successful
0926 06:20:02.217181 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":73,"threads_started":1}
I20260926 06:20:02.217255 7251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:02.213441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:59409 (local address 127.4.202.129:33965)
0926 06:20:02.213585 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:02.213587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:02.213610 (+ 23us) server_negotiation.cc:407] Connection header received
0926 06:20:02.213640 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:02.213641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:02.213652 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:02.213735 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:20:02.214009 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:02.214783 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:02.216213 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:02.216824 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:02.216945 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:02.216953 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:02.217135 (+ 182us) server_negotiation.cc:299] Negotiation successful
0926 06:20:02.217184 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
I20260926 06:20:02.217298 7249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:02.213287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:33965 (local address 127.4.202.131:59409)
0926 06:20:02.213466 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:02.213478 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:02.213571 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:02.213751 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:02.213752 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:02.213754 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:20:02.213926 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:02.213928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:02.214860 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:02.214861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:02.216117 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:02.216120 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:02.216942 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:02.216943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:02.217033 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:02.217036 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:20:02.217091 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 06:20:02.217148 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":2,"thread_start_us":133,"threads_started":1}
I20260926 06:20:02.217303 7248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:02.213287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:43075 (local address 127.4.202.131:48063)
0926 06:20:02.213512 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:02.213524 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:02.213585 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:02.213789 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:02.213790 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:02.213792 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:20:02.214239 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:02.214242 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:02.215070 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:02.215071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:02.216198 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:02.216200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:02.216921 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:02.216923 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:02.217017 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:02.217022 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 06:20:02.217091 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 06:20:02.217148 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":26,"thread_start_us":96,"threads_started":1}
I20260926 06:20:02.217854 6801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "897704d052154173944c901b5026d1ed" candidate_uuid: "0bab59c2bae246d3a307d3d8af888422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233d6dedde314417b8658dadb05c360b" is_pre_election: true
I20260926 06:20:02.217924 6942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "897704d052154173944c901b5026d1ed" candidate_uuid: "0bab59c2bae246d3a307d3d8af888422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17bfeab63b3149a1bdedc53208a4bf57" is_pre_election: true
I20260926 06:20:02.217968 6801 raft_consensus.cc:2454] T 897704d052154173944c901b5026d1ed P 233d6dedde314417b8658dadb05c360b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bab59c2bae246d3a307d3d8af888422 in term 0.
I20260926 06:20:02.218001 6942 raft_consensus.cc:2454] T 897704d052154173944c901b5026d1ed P 17bfeab63b3149a1bdedc53208a4bf57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bab59c2bae246d3a307d3d8af888422 in term 0.
I20260926 06:20:02.218147 7014 leader_election.cc:302] T 897704d052154173944c901b5026d1ed P 0bab59c2bae246d3a307d3d8af888422 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bab59c2bae246d3a307d3d8af888422, 233d6dedde314417b8658dadb05c360b; no voters:
I20260926 06:20:02.218281 7247 raft_consensus.cc:2781] T 897704d052154173944c901b5026d1ed P 0bab59c2bae246d3a307d3d8af888422 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:20:02.218318 7247 raft_consensus.cc:482] T 897704d052154173944c901b5026d1ed P 0bab59c2bae246d3a307d3d8af888422 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:20:02.218341 7247 raft_consensus.cc:3037] T 897704d052154173944c901b5026d1ed P 0bab59c2bae246d3a307d3d8af888422 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:02.219128 7247 raft_consensus.cc:504] T 897704d052154173944c901b5026d1ed P 0bab59c2bae246d3a307d3d8af888422 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:02.219278 7247 leader_election.cc:288] T 897704d052154173944c901b5026d1ed P 0bab59c2bae246d3a307d3d8af888422 [CANDIDATE]: Term 1 election: Requested vote from peers 233d6dedde314417b8658dadb05c360b (127.4.202.129:33965), 17bfeab63b3149a1bdedc53208a4bf57 (127.4.202.130:43075)
I20260926 06:20:02.219393 6801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "897704d052154173944c901b5026d1ed" candidate_uuid: "0bab59c2bae246d3a307d3d8af888422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233d6dedde314417b8658dadb05c360b"
I20260926 06:20:02.219416 6942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "897704d052154173944c901b5026d1ed" candidate_uuid: "0bab59c2bae246d3a307d3d8af888422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17bfeab63b3149a1bdedc53208a4bf57"
I20260926 06:20:02.219472 6801 raft_consensus.cc:3037] T 897704d052154173944c901b5026d1ed P 233d6dedde314417b8658dadb05c360b [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:02.219514 6942 raft_consensus.cc:3037] T 897704d052154173944c901b5026d1ed P 17bfeab63b3149a1bdedc53208a4bf57 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:02.220263 6801 raft_consensus.cc:2454] T 897704d052154173944c901b5026d1ed P 233d6dedde314417b8658dadb05c360b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bab59c2bae246d3a307d3d8af888422 in term 1.
I20260926 06:20:02.220263 6942 raft_consensus.cc:2454] T 897704d052154173944c901b5026d1ed P 17bfeab63b3149a1bdedc53208a4bf57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bab59c2bae246d3a307d3d8af888422 in term 1.
I20260926 06:20:02.220440 7014 leader_election.cc:302] T 897704d052154173944c901b5026d1ed P 0bab59c2bae246d3a307d3d8af888422 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bab59c2bae246d3a307d3d8af888422, 233d6dedde314417b8658dadb05c360b; no voters:
I20260926 06:20:02.220579 7247 raft_consensus.cc:2781] T 897704d052154173944c901b5026d1ed P 0bab59c2bae246d3a307d3d8af888422 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:20:02.220712 7247 raft_consensus.cc:686] T 897704d052154173944c901b5026d1ed P 0bab59c2bae246d3a307d3d8af888422 [term 1 LEADER]: Becoming Leader. State: Replica: 0bab59c2bae246d3a307d3d8af888422, State: Running, Role: LEADER
I20260926 06:20:02.220763 7247 consensus_queue.cc:237] T 897704d052154173944c901b5026d1ed P 0bab59c2bae246d3a307d3d8af888422 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:02.221432 6643 catalog_manager.cc:5495] T 897704d052154173944c901b5026d1ed P 0bab59c2bae246d3a307d3d8af888422 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bab59c2bae246d3a307d3d8af888422 (127.4.202.131). New cstate: current_term: 1 leader_uuid: "0bab59c2bae246d3a307d3d8af888422" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } health_report { overall_health: HEALTHY } } }
I20260926 06:20:02.248838 7255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:02.246384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54094 (local address 127.4.202.131:39917)
0926 06:20:02.246561 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:02.246563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:02.246568 (+ 5us) server_negotiation.cc:407] Connection header received
0926 06:20:02.246610 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:02.246611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:02.246626 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:02.246685 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:20:02.246809 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:02.247363 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:02.248137 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:02.248374 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:02.248424 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:02.248564 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:20:02.248657 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:20:02.248711 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:02.248756 (+ 45us) server_negotiation.cc:299] Negotiation successful
0926 06:20:02.248771 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":98,"threads_started":1}
I20260926 06:20:02.253288 7250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:02.251018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:35243 (local address 127.4.202.130:43075)
0926 06:20:02.251080 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:02.251082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:02.251091 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:20:02.251158 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:02.251159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:02.251167 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:02.251233 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:20:02.251369 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:02.251827 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:02.252604 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:02.252801 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:02.252873 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:02.253049 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:20:02.253086 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:20:02.253151 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:02.253203 (+ 52us) server_negotiation.cc:299] Negotiation successful
0926 06:20:02.253222 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 06:20:02.253328 7258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:02.250893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:43075 (local address 127.4.202.131:35243)
0926 06:20:02.251039 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:02.251051 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:02.251101 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:02.251235 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:02.251236 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:02.251237 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 06:20:02.251305 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:02.251306 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:02.251904 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:02.251905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:02.252516 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:02.252517 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:02.252883 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:02.252884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:02.252951 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:02.252962 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 06:20:02.253153 (+ 191us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 06:20:02.253155 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:20:02.253202 (+ 47us) client_negotiation.cc:240] Negotiation successful
0926 06:20:02.253241 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
I20260926 06:20:02.255159 6942 raft_consensus.cc:1261] T 897704d052154173944c901b5026d1ed P 17bfeab63b3149a1bdedc53208a4bf57 [term 1 FOLLOWER]: Refusing update from remote peer 0bab59c2bae246d3a307d3d8af888422: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:20:02.255234 6801 raft_consensus.cc:1261] T 897704d052154173944c901b5026d1ed P 233d6dedde314417b8658dadb05c360b [term 1 FOLLOWER]: Refusing update from remote peer 0bab59c2bae246d3a307d3d8af888422: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:20:02.255467 7252 consensus_queue.cc:1035] T 897704d052154173944c901b5026d1ed P 0bab59c2bae246d3a307d3d8af888422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:20:02.255577 7247 consensus_queue.cc:1035] T 897704d052154173944c901b5026d1ed P 0bab59c2bae246d3a307d3d8af888422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:20:02.261747 7255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:02.259895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:35489 (local address 127.4.202.131:39917)
0926 06:20:02.259933 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:02.259935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:02.259942 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:20:02.260002 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:02.260003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:02.260012 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:02.260063 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:20:02.260172 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:02.260572 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:02.261198 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:02.261356 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:02.261402 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:02.261556 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:20:02.261585 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:20:02.261635 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:02.261694 (+ 59us) server_negotiation.cc:299] Negotiation successful
0926 06:20:02.261708 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 06:20:02.261838 7263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:02.259745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:39917 (local address 127.4.202.130:35489)
0926 06:20:02.259893 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:02.259902 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:02.259954 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:02.260068 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:02.260069 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:02.260071 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 06:20:02.260114 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:02.260115 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:02.260632 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:02.260633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:02.261134 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:02.261135 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:02.261417 (+ 282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:02.261418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:02.261474 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:02.261479 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 06:20:02.261647 (+ 168us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 06:20:02.261648 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 06:20:02.261694 (+ 46us) client_negotiation.cc:240] Negotiation successful
0926 06:20:02.261737 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":67,"threads_started":1}
I20260926 06:20:02.269817 6643 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:20:02.269913 6643 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:20:02.270537 6643 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4707268ef11e42139ecf487c90adb27a: Sending DeleteTablet for 3 replicas of tablet 897704d052154173944c901b5026d1ed
I20260926 06:20:02.270779 6782 tablet_service.cc:1514] Processing DeleteTablet for tablet 897704d052154173944c901b5026d1ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:20:02 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:58658
I20260926 06:20:02.270797 6922 tablet_service.cc:1514] Processing DeleteTablet for tablet 897704d052154173944c901b5026d1ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:20:02 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:42460
I20260926 06:20:02.270823 7062 tablet_service.cc:1514] Processing DeleteTablet for tablet 897704d052154173944c901b5026d1ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:20:02 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:54084
I20260926 06:20:02.270875 7230 tablet_replica.cc:330] T 897704d052154173944c901b5026d1ed P 233d6dedde314417b8658dadb05c360b: stopping tablet replica
I20260926 06:20:02.270879 7232 tablet_replica.cc:330] T 897704d052154173944c901b5026d1ed P 17bfeab63b3149a1bdedc53208a4bf57: stopping tablet replica
I20260926 06:20:02.270903 7231 tablet_replica.cc:330] T 897704d052154173944c901b5026d1ed P 0bab59c2bae246d3a307d3d8af888422: stopping tablet replica
I20260926 06:20:02.270931 7232 raft_consensus.cc:2229] T 897704d052154173944c901b5026d1ed P 17bfeab63b3149a1bdedc53208a4bf57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:20:02.270933 7230 raft_consensus.cc:2229] T 897704d052154173944c901b5026d1ed P 233d6dedde314417b8658dadb05c360b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:20:02.270948 7231 raft_consensus.cc:2229] T 897704d052154173944c901b5026d1ed P 0bab59c2bae246d3a307d3d8af888422 [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:20:02.270972 7230 raft_consensus.cc:2258] T 897704d052154173944c901b5026d1ed P 233d6dedde314417b8658dadb05c360b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:20:02.270977 7232 raft_consensus.cc:2258] T 897704d052154173944c901b5026d1ed P 17bfeab63b3149a1bdedc53208a4bf57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:20:02.271019 7231 raft_consensus.cc:2258] T 897704d052154173944c901b5026d1ed P 0bab59c2bae246d3a307d3d8af888422 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:20:02.271310 7232 ts_tablet_manager.cc:1905] T 897704d052154173944c901b5026d1ed P 17bfeab63b3149a1bdedc53208a4bf57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:20:02.271376 7230 ts_tablet_manager.cc:1905] T 897704d052154173944c901b5026d1ed P 233d6dedde314417b8658dadb05c360b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:20:02.272586 7231 ts_tablet_manager.cc:1905] T 897704d052154173944c901b5026d1ed P 0bab59c2bae246d3a307d3d8af888422: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:20:02.272739 7232 ts_tablet_manager.cc:1918] T 897704d052154173944c901b5026d1ed P 17bfeab63b3149a1bdedc53208a4bf57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 06:20:02.272794 7232 log.cc:1196] T 897704d052154173944c901b5026d1ed P 17bfeab63b3149a1bdedc53208a4bf57: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-1/wal/wals/897704d052154173944c901b5026d1ed
I20260926 06:20:02.272745 6643 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36544:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:20:02.272863 6643 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:20:02.272866 7230 ts_tablet_manager.cc:1918] T 897704d052154173944c901b5026d1ed P 233d6dedde314417b8658dadb05c360b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 06:20:02.272913 7230 log.cc:1196] T 897704d052154173944c901b5026d1ed P 233d6dedde314417b8658dadb05c360b: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-0/wal/wals/897704d052154173944c901b5026d1ed
I20260926 06:20:02.273018 7232 ts_tablet_manager.cc:1939] T 897704d052154173944c901b5026d1ed P 17bfeab63b3149a1bdedc53208a4bf57: Deleting consensus metadata
I20260926 06:20:02.273133 7230 ts_tablet_manager.cc:1939] T 897704d052154173944c901b5026d1ed P 233d6dedde314417b8658dadb05c360b: Deleting consensus metadata
I20260926 06:20:02.273406 6631 catalog_manager.cc:4841] TS 17bfeab63b3149a1bdedc53208a4bf57 (127.4.202.130:43075): tablet 897704d052154173944c901b5026d1ed (table test-table [id=b5131d90486649f9bb8f03de1f96b593]) successfully deleted
I20260926 06:20:02.273675 6630 catalog_manager.cc:4841] TS 233d6dedde314417b8658dadb05c360b (127.4.202.129:33965): tablet 897704d052154173944c901b5026d1ed (table test-table [id=b5131d90486649f9bb8f03de1f96b593]) successfully deleted
I20260926 06:20:02.273922 7231 ts_tablet_manager.cc:1918] T 897704d052154173944c901b5026d1ed P 0bab59c2bae246d3a307d3d8af888422: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 06:20:02.273959 7231 log.cc:1196] T 897704d052154173944c901b5026d1ed P 0bab59c2bae246d3a307d3d8af888422: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-2/wal/wals/897704d052154173944c901b5026d1ed
I20260926 06:20:02.274107 7231 ts_tablet_manager.cc:1939] T 897704d052154173944c901b5026d1ed P 0bab59c2bae246d3a307d3d8af888422: Deleting consensus metadata
I20260926 06:20:02.274842 6632 catalog_manager.cc:4841] TS 0bab59c2bae246d3a307d3d8af888422 (127.4.202.131:39917): tablet 897704d052154173944c901b5026d1ed (table test-table [id=b5131d90486649f9bb8f03de1f96b593]) successfully deleted
I20260926 06:20:02.275269 6782 tablet_service.cc:1467] Processing CreateTablet for tablet 9496d8a14eb34318acb729e67412cc06 (DEFAULT_TABLE table=test-table [id=7802eebef69a4282af73fcdfa75fdc4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:20:02.275265 7062 tablet_service.cc:1467] Processing CreateTablet for tablet 9496d8a14eb34318acb729e67412cc06 (DEFAULT_TABLE table=test-table [id=7802eebef69a4282af73fcdfa75fdc4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:20:02.275269 6922 tablet_service.cc:1467] Processing CreateTablet for tablet 9496d8a14eb34318acb729e67412cc06 (DEFAULT_TABLE table=test-table [id=7802eebef69a4282af73fcdfa75fdc4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:20:02.275367 7062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9496d8a14eb34318acb729e67412cc06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:02.275365 6782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9496d8a14eb34318acb729e67412cc06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:02.275368 6922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9496d8a14eb34318acb729e67412cc06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:02.276470 7244 tablet_bootstrap.cc:492] T 9496d8a14eb34318acb729e67412cc06 P 0bab59c2bae246d3a307d3d8af888422: Bootstrap starting.
I20260926 06:20:02.276522 7241 tablet_bootstrap.cc:492] T 9496d8a14eb34318acb729e67412cc06 P 233d6dedde314417b8658dadb05c360b: Bootstrap starting.
I20260926 06:20:02.276566 7242 tablet_bootstrap.cc:492] T 9496d8a14eb34318acb729e67412cc06 P 17bfeab63b3149a1bdedc53208a4bf57: Bootstrap starting.
I20260926 06:20:02.276822 7242 tablet_bootstrap.cc:654] T 9496d8a14eb34318acb729e67412cc06 P 17bfeab63b3149a1bdedc53208a4bf57: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:02.277235 7241 tablet_bootstrap.cc:654] T 9496d8a14eb34318acb729e67412cc06 P 233d6dedde314417b8658dadb05c360b: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:02.277297 7244 tablet_bootstrap.cc:654] T 9496d8a14eb34318acb729e67412cc06 P 0bab59c2bae246d3a307d3d8af888422: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:02.277846 7241 tablet_bootstrap.cc:492] T 9496d8a14eb34318acb729e67412cc06 P 233d6dedde314417b8658dadb05c360b: No bootstrap required, opened a new log
I20260926 06:20:02.277904 7241 ts_tablet_manager.cc:1397] T 9496d8a14eb34318acb729e67412cc06 P 233d6dedde314417b8658dadb05c360b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 06:20:02.277997 7242 tablet_bootstrap.cc:492] T 9496d8a14eb34318acb729e67412cc06 P 17bfeab63b3149a1bdedc53208a4bf57: No bootstrap required, opened a new log
I20260926 06:20:02.278043 7242 ts_tablet_manager.cc:1397] T 9496d8a14eb34318acb729e67412cc06 P 17bfeab63b3149a1bdedc53208a4bf57: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 06:20:02.278041 7241 raft_consensus.cc:348] T 9496d8a14eb34318acb729e67412cc06 P 233d6dedde314417b8658dadb05c360b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:02.278105 7241 raft_consensus.cc:374] T 9496d8a14eb34318acb729e67412cc06 P 233d6dedde314417b8658dadb05c360b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:02.278124 7241 raft_consensus.cc:729] T 9496d8a14eb34318acb729e67412cc06 P 233d6dedde314417b8658dadb05c360b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233d6dedde314417b8658dadb05c360b, State: Initialized, Role: FOLLOWER
I20260926 06:20:02.278147 7242 raft_consensus.cc:348] T 9496d8a14eb34318acb729e67412cc06 P 17bfeab63b3149a1bdedc53208a4bf57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:02.278190 7241 consensus_queue.cc:260] T 9496d8a14eb34318acb729e67412cc06 P 233d6dedde314417b8658dadb05c360b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:02.278219 7242 raft_consensus.cc:374] T 9496d8a14eb34318acb729e67412cc06 P 17bfeab63b3149a1bdedc53208a4bf57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:02.278245 7242 raft_consensus.cc:729] T 9496d8a14eb34318acb729e67412cc06 P 17bfeab63b3149a1bdedc53208a4bf57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17bfeab63b3149a1bdedc53208a4bf57, State: Initialized, Role: FOLLOWER
I20260926 06:20:02.278290 7241 ts_tablet_manager.cc:1428] T 9496d8a14eb34318acb729e67412cc06 P 233d6dedde314417b8658dadb05c360b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:02.278293 7242 consensus_queue.cc:260] T 9496d8a14eb34318acb729e67412cc06 P 17bfeab63b3149a1bdedc53208a4bf57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:02.278388 7242 ts_tablet_manager.cc:1428] T 9496d8a14eb34318acb729e67412cc06 P 17bfeab63b3149a1bdedc53208a4bf57: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:02.278846 7244 tablet_bootstrap.cc:492] T 9496d8a14eb34318acb729e67412cc06 P 0bab59c2bae246d3a307d3d8af888422: No bootstrap required, opened a new log
I20260926 06:20:02.278908 7244 ts_tablet_manager.cc:1397] T 9496d8a14eb34318acb729e67412cc06 P 0bab59c2bae246d3a307d3d8af888422: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 06:20:02.279070 7244 raft_consensus.cc:348] T 9496d8a14eb34318acb729e67412cc06 P 0bab59c2bae246d3a307d3d8af888422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:02.279135 7244 raft_consensus.cc:374] T 9496d8a14eb34318acb729e67412cc06 P 0bab59c2bae246d3a307d3d8af888422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:02.279150 7244 raft_consensus.cc:729] T 9496d8a14eb34318acb729e67412cc06 P 0bab59c2bae246d3a307d3d8af888422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bab59c2bae246d3a307d3d8af888422, State: Initialized, Role: FOLLOWER
I20260926 06:20:02.279196 7244 consensus_queue.cc:260] T 9496d8a14eb34318acb729e67412cc06 P 0bab59c2bae246d3a307d3d8af888422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:02.279301 7244 ts_tablet_manager.cc:1428] T 9496d8a14eb34318acb729e67412cc06 P 0bab59c2bae246d3a307d3d8af888422: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:02.463938 7261 raft_consensus.cc:482] T 9496d8a14eb34318acb729e67412cc06 P 0bab59c2bae246d3a307d3d8af888422 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:20:02.464028 7261 raft_consensus.cc:504] T 9496d8a14eb34318acb729e67412cc06 P 0bab59c2bae246d3a307d3d8af888422 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:02.464197 7261 leader_election.cc:288] T 9496d8a14eb34318acb729e67412cc06 P 0bab59c2bae246d3a307d3d8af888422 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233d6dedde314417b8658dadb05c360b (127.4.202.129:33965), 17bfeab63b3149a1bdedc53208a4bf57 (127.4.202.130:43075)
I20260926 06:20:02.464462 6801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9496d8a14eb34318acb729e67412cc06" candidate_uuid: "0bab59c2bae246d3a307d3d8af888422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233d6dedde314417b8658dadb05c360b" is_pre_election: true
I20260926 06:20:02.464462 6942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9496d8a14eb34318acb729e67412cc06" candidate_uuid: "0bab59c2bae246d3a307d3d8af888422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17bfeab63b3149a1bdedc53208a4bf57" is_pre_election: true
I20260926 06:20:02.464578 6801 raft_consensus.cc:2454] T 9496d8a14eb34318acb729e67412cc06 P 233d6dedde314417b8658dadb05c360b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bab59c2bae246d3a307d3d8af888422 in term 0.
I20260926 06:20:02.464578 6942 raft_consensus.cc:2454] T 9496d8a14eb34318acb729e67412cc06 P 17bfeab63b3149a1bdedc53208a4bf57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bab59c2bae246d3a307d3d8af888422 in term 0.
I20260926 06:20:02.464769 7015 leader_election.cc:302] T 9496d8a14eb34318acb729e67412cc06 P 0bab59c2bae246d3a307d3d8af888422 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bab59c2bae246d3a307d3d8af888422, 17bfeab63b3149a1bdedc53208a4bf57; no voters:
I20260926 06:20:02.464908 7261 raft_consensus.cc:2781] T 9496d8a14eb34318acb729e67412cc06 P 0bab59c2bae246d3a307d3d8af888422 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:20:02.464983 7261 raft_consensus.cc:482] T 9496d8a14eb34318acb729e67412cc06 P 0bab59c2bae246d3a307d3d8af888422 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:20:02.465018 7261 raft_consensus.cc:3037] T 9496d8a14eb34318acb729e67412cc06 P 0bab59c2bae246d3a307d3d8af888422 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:02.465727 7261 raft_consensus.cc:504] T 9496d8a14eb34318acb729e67412cc06 P 0bab59c2bae246d3a307d3d8af888422 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:02.465898 7261 leader_election.cc:288] T 9496d8a14eb34318acb729e67412cc06 P 0bab59c2bae246d3a307d3d8af888422 [CANDIDATE]: Term 1 election: Requested vote from peers 233d6dedde314417b8658dadb05c360b (127.4.202.129:33965), 17bfeab63b3149a1bdedc53208a4bf57 (127.4.202.130:43075)
I20260926 06:20:02.466066 6801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9496d8a14eb34318acb729e67412cc06" candidate_uuid: "0bab59c2bae246d3a307d3d8af888422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233d6dedde314417b8658dadb05c360b"
I20260926 06:20:02.466073 6942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9496d8a14eb34318acb729e67412cc06" candidate_uuid: "0bab59c2bae246d3a307d3d8af888422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17bfeab63b3149a1bdedc53208a4bf57"
I20260926 06:20:02.466146 6942 raft_consensus.cc:3037] T 9496d8a14eb34318acb729e67412cc06 P 17bfeab63b3149a1bdedc53208a4bf57 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:02.466146 6801 raft_consensus.cc:3037] T 9496d8a14eb34318acb729e67412cc06 P 233d6dedde314417b8658dadb05c360b [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:02.466920 6801 raft_consensus.cc:2454] T 9496d8a14eb34318acb729e67412cc06 P 233d6dedde314417b8658dadb05c360b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bab59c2bae246d3a307d3d8af888422 in term 1.
I20260926 06:20:02.466920 6942 raft_consensus.cc:2454] T 9496d8a14eb34318acb729e67412cc06 P 17bfeab63b3149a1bdedc53208a4bf57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bab59c2bae246d3a307d3d8af888422 in term 1.
I20260926 06:20:02.467136 7014 leader_election.cc:302] T 9496d8a14eb34318acb729e67412cc06 P 0bab59c2bae246d3a307d3d8af888422 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bab59c2bae246d3a307d3d8af888422, 233d6dedde314417b8658dadb05c360b; no voters:
I20260926 06:20:02.467270 7261 raft_consensus.cc:2781] T 9496d8a14eb34318acb729e67412cc06 P 0bab59c2bae246d3a307d3d8af888422 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:20:02.467345 7261 raft_consensus.cc:686] T 9496d8a14eb34318acb729e67412cc06 P 0bab59c2bae246d3a307d3d8af888422 [term 1 LEADER]: Becoming Leader. State: Replica: 0bab59c2bae246d3a307d3d8af888422, State: Running, Role: LEADER
I20260926 06:20:02.467397 7261 consensus_queue.cc:237] T 9496d8a14eb34318acb729e67412cc06 P 0bab59c2bae246d3a307d3d8af888422 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } }
I20260926 06:20:02.468088 6643 catalog_manager.cc:5495] T 9496d8a14eb34318acb729e67412cc06 P 0bab59c2bae246d3a307d3d8af888422 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bab59c2bae246d3a307d3d8af888422 (127.4.202.131). New cstate: current_term: 1 leader_uuid: "0bab59c2bae246d3a307d3d8af888422" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bab59c2bae246d3a307d3d8af888422" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 39917 } health_report { overall_health: HEALTHY } } }
I20260926 06:20:02.506711 6801 raft_consensus.cc:1261] T 9496d8a14eb34318acb729e67412cc06 P 233d6dedde314417b8658dadb05c360b [term 1 FOLLOWER]: Refusing update from remote peer 0bab59c2bae246d3a307d3d8af888422: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:20:02.506695 6942 raft_consensus.cc:1261] T 9496d8a14eb34318acb729e67412cc06 P 17bfeab63b3149a1bdedc53208a4bf57 [term 1 FOLLOWER]: Refusing update from remote peer 0bab59c2bae246d3a307d3d8af888422: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:20:02.506899 7261 consensus_queue.cc:1035] T 9496d8a14eb34318acb729e67412cc06 P 0bab59c2bae246d3a307d3d8af888422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17bfeab63b3149a1bdedc53208a4bf57" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:20:02.506999 7252 consensus_queue.cc:1035] T 9496d8a14eb34318acb729e67412cc06 P 0bab59c2bae246d3a307d3d8af888422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "233d6dedde314417b8658dadb05c360b" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 06:20:02.510788 7212 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260926 06:20:04.027971 6643 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:20:04.028075 6643 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:20:04.028844 6643 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4707268ef11e42139ecf487c90adb27a: Sending DeleteTablet for 3 replicas of tablet 9496d8a14eb34318acb729e67412cc06
I20260926 06:20:04.029100 6782 tablet_service.cc:1514] Processing DeleteTablet for tablet 9496d8a14eb34318acb729e67412cc06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:20:04 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:58658
I20260926 06:20:04.029127 7062 tablet_service.cc:1514] Processing DeleteTablet for tablet 9496d8a14eb34318acb729e67412cc06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:20:04 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:54084
I20260926 06:20:04.029239 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 6717
I20260926 06:20:04.029286 6922 tablet_service.cc:1514] Processing DeleteTablet for tablet 9496d8a14eb34318acb729e67412cc06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:20:04 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:42460
I20260926 06:20:04.030768 7295 tablet_replica.cc:330] T 9496d8a14eb34318acb729e67412cc06 P 17bfeab63b3149a1bdedc53208a4bf57: stopping tablet replica
I20260926 06:20:04.030859 7295 raft_consensus.cc:2229] T 9496d8a14eb34318acb729e67412cc06 P 17bfeab63b3149a1bdedc53208a4bf57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:20:04.030923 7295 raft_consensus.cc:2258] T 9496d8a14eb34318acb729e67412cc06 P 17bfeab63b3149a1bdedc53208a4bf57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:20:04.031186 7295 ts_tablet_manager.cc:1905] T 9496d8a14eb34318acb729e67412cc06 P 17bfeab63b3149a1bdedc53208a4bf57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:20:04.032366 7294 tablet_replica.cc:330] T 9496d8a14eb34318acb729e67412cc06 P 0bab59c2bae246d3a307d3d8af888422: stopping tablet replica
I20260926 06:20:04.032421 7294 raft_consensus.cc:2229] T 9496d8a14eb34318acb729e67412cc06 P 0bab59c2bae246d3a307d3d8af888422 [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:20:04.032476 7294 raft_consensus.cc:2258] T 9496d8a14eb34318acb729e67412cc06 P 0bab59c2bae246d3a307d3d8af888422 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:20:04.032728 7295 ts_tablet_manager.cc:1918] T 9496d8a14eb34318acb729e67412cc06 P 17bfeab63b3149a1bdedc53208a4bf57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 06:20:04.032791 7295 log.cc:1196] T 9496d8a14eb34318acb729e67412cc06 P 17bfeab63b3149a1bdedc53208a4bf57: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-1/wal/wals/9496d8a14eb34318acb729e67412cc06
I20260926 06:20:04.032771 7294 ts_tablet_manager.cc:1905] T 9496d8a14eb34318acb729e67412cc06 P 0bab59c2bae246d3a307d3d8af888422: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:20:04.033044 7295 ts_tablet_manager.cc:1939] T 9496d8a14eb34318acb729e67412cc06 P 17bfeab63b3149a1bdedc53208a4bf57: Deleting consensus metadata
I20260926 06:20:04.033380 6631 catalog_manager.cc:4841] TS 17bfeab63b3149a1bdedc53208a4bf57 (127.4.202.130:43075): tablet 9496d8a14eb34318acb729e67412cc06 (table test-table [id=7802eebef69a4282af73fcdfa75fdc4d]) successfully deleted
I20260926 06:20:04.034416 7294 ts_tablet_manager.cc:1918] T 9496d8a14eb34318acb729e67412cc06 P 0bab59c2bae246d3a307d3d8af888422: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 06:20:04.034495 7294 log.cc:1196] T 9496d8a14eb34318acb729e67412cc06 P 0bab59c2bae246d3a307d3d8af888422: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790403591640726-4906-0/minicluster-data/ts-2/wal/wals/9496d8a14eb34318acb729e67412cc06
I20260926 06:20:04.034771 7294 ts_tablet_manager.cc:1939] T 9496d8a14eb34318acb729e67412cc06 P 0bab59c2bae246d3a307d3d8af888422: Deleting consensus metadata
I20260926 06:20:04.035092 6632 catalog_manager.cc:4841] TS 0bab59c2bae246d3a307d3d8af888422 (127.4.202.131:39917): tablet 9496d8a14eb34318acb729e67412cc06 (table test-table [id=7802eebef69a4282af73fcdfa75fdc4d]) successfully deleted
W20260926 06:20:04.036906 6630 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.202.129:33965 (error 108)
I20260926 06:20:04.037113 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 6857
I20260926 06:20:04.042673 7297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:04.037267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:33965 (local address 127.0.0.1:58678)
0926 06:20:04.042548 (+ 5281us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:04.042633 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.129:33965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5248,"thread_start_us":57,"threads_started":1}
I20260926 06:20:04.042838 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 6997
W20260926 06:20:04.042829 6630 catalog_manager.cc:4584] TS 233d6dedde314417b8658dadb05c360b (127.4.202.129:33965): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9496d8a14eb34318acb729e67412cc06: Network error: Client connection negotiation failed: client connection to 127.4.202.129:33965: connect: Connection refused (error 111)
I20260926 06:20:04.047643 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 6613
2026-09-26T06:20:04Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6047 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:20:04 dist-test-slave-pd0t krb5kdc[7300](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:20:04 dist-test-slave-pd0t krb5kdc[7300](info): set up 2 sockets
Sep 26 06:20:04 dist-test-slave-pd0t krb5kdc[7300](info): commencing operation
krb5kdc: starting...
W20260926 06:20:05.066242 4906 mini_kdc.cc:121] Time spent starting KDC: real 0.998s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:20:05 dist-test-slave-pd0t krb5kdc[7300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:20:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:20:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.202.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
I20260926 06:20:05.177498 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:46843
--webserver_interface=127.4.202.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:35221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:46843
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:05.253597 7313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:05.253786 7313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:05.253806 7313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:05.254944 7313 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:20:05.254983 7313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:05.254998 7313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:05.255010 7313 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:20:05.255023 7313 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:20:05.256354 7313 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:35221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:46843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:46843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:05.256521 7313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:05.256665 7313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:05.261353 7318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:05.261360 7321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:05.261415 7313 server_base.cc:1034] running on GCE node
W20260926 06:20:05.261379 7319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:05.261835 7313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:05.262112 7313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:05.263247 7313 hybrid_clock.cc:648] HybridClock initialized: now 1790403605263226 us; error 36 us; skew 500 ppm
Sep 26 06:20:05 dist-test-slave-pd0t krb5kdc[7300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403605, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:05.270808 7313 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:20:05.271255 7313 webserver.cc:460] Webserver started at http://127.4.202.190:34707/ using document root <none> and password file <none>
I20260926 06:20:05.271476 7313 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:05.271520 7313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:05.271644 7313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:05.272677 7313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "0c56fbeb893a4bf185d475fad83f78cb"
format_stamp: "Formatted at 2026-09-26 06:20:05 on dist-test-slave-pd0t"
server_key: "056b2444b2a03912d84f64c86ee398bd"
server_key_iv: "2f3c4ea34ac485d0228b6a7d1744e36e"
server_key_version: "encryptionkey@0"
I20260926 06:20:05.273027 7313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "0c56fbeb893a4bf185d475fad83f78cb"
format_stamp: "Formatted at 2026-09-26 06:20:05 on dist-test-slave-pd0t"
server_key: "056b2444b2a03912d84f64c86ee398bd"
server_key_iv: "2f3c4ea34ac485d0228b6a7d1744e36e"
server_key_version: "encryptionkey@0"
I20260926 06:20:05.274255 7313 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:20:05.275081 7328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:05.275251 7313 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:05.275331 7313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "0c56fbeb893a4bf185d475fad83f78cb"
format_stamp: "Formatted at 2026-09-26 06:20:05 on dist-test-slave-pd0t"
server_key: "056b2444b2a03912d84f64c86ee398bd"
server_key_iv: "2f3c4ea34ac485d0228b6a7d1744e36e"
server_key_version: "encryptionkey@0"
I20260926 06:20:05.275408 7313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:05.299894 7313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:05.302057 7313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:05.302217 7313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:05.306095 7313 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:46843
I20260926 06:20:05.306166 7380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:46843 every 8 connection(s)
I20260926 06:20:05.306478 7313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:20:05.307337 7381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:05.309463 7381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c56fbeb893a4bf185d475fad83f78cb: Bootstrap starting.
I20260926 06:20:05.309970 7381 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c56fbeb893a4bf185d475fad83f78cb: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:05.310205 7381 log.cc:824] T 00000000000000000000000000000000 P 0c56fbeb893a4bf185d475fad83f78cb: Log is configured to *not* fsync() on all Append() calls
I20260926 06:20:05.310842 7381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c56fbeb893a4bf185d475fad83f78cb: No bootstrap required, opened a new log
I20260926 06:20:05.311642 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 7313
I20260926 06:20:05.311743 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:20:05.311910 4906 external_mini_cluster.cc:1428] Setting key 2f410e6e988a1338f2654ee244c9b297
I20260926 06:20:05.312750 7381 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c56fbeb893a4bf185d475fad83f78cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c56fbeb893a4bf185d475fad83f78cb" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 46843 } }
I20260926 06:20:05.312922 7381 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c56fbeb893a4bf185d475fad83f78cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:05.312961 7381 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c56fbeb893a4bf185d475fad83f78cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c56fbeb893a4bf185d475fad83f78cb, State: Initialized, Role: FOLLOWER
I20260926 06:20:05.313125 7381 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c56fbeb893a4bf185d475fad83f78cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c56fbeb893a4bf185d475fad83f78cb" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 46843 } }
I20260926 06:20:05.313220 7381 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c56fbeb893a4bf185d475fad83f78cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:20:05.313256 7381 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c56fbeb893a4bf185d475fad83f78cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:20:05.313309 7381 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c56fbeb893a4bf185d475fad83f78cb [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:05.314265 7381 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c56fbeb893a4bf185d475fad83f78cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c56fbeb893a4bf185d475fad83f78cb" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 46843 } }
I20260926 06:20:05.314352 7381 leader_election.cc:302] T 00000000000000000000000000000000 P 0c56fbeb893a4bf185d475fad83f78cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c56fbeb893a4bf185d475fad83f78cb; no voters:
I20260926 06:20:05.314523 7381 leader_election.cc:288] T 00000000000000000000000000000000 P 0c56fbeb893a4bf185d475fad83f78cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:20:05.314564 7386 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c56fbeb893a4bf185d475fad83f78cb [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:20:05.314730 7386 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c56fbeb893a4bf185d475fad83f78cb [term 1 LEADER]: Becoming Leader. State: Replica: 0c56fbeb893a4bf185d475fad83f78cb, State: Running, Role: LEADER
I20260926 06:20:05.314798 7381 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c56fbeb893a4bf185d475fad83f78cb [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:20:05.314857 7386 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c56fbeb893a4bf185d475fad83f78cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c56fbeb893a4bf185d475fad83f78cb" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 46843 } }
I20260926 06:20:05.315168 7386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c56fbeb893a4bf185d475fad83f78cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c56fbeb893a4bf185d475fad83f78cb. Latest consensus state: current_term: 1 leader_uuid: "0c56fbeb893a4bf185d475fad83f78cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c56fbeb893a4bf185d475fad83f78cb" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 46843 } } }
I20260926 06:20:05.315330 7386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c56fbeb893a4bf185d475fad83f78cb [sys.catalog]: This master's current role is: LEADER
I20260926 06:20:05.315590 7394 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:20:05.316035 7387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c56fbeb893a4bf185d475fad83f78cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c56fbeb893a4bf185d475fad83f78cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c56fbeb893a4bf185d475fad83f78cb" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 46843 } } }
I20260926 06:20:05.316154 7387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c56fbeb893a4bf185d475fad83f78cb [sys.catalog]: This master's current role is: LEADER
I20260926 06:20:05.316092 7394 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:20:05.318272 7394 catalog_manager.cc:1347] Generated new cluster ID: 159f76dbf44f47639a61b6ac3a296d64
I20260926 06:20:05.318336 7394 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:20:05 dist-test-slave-pd0t krb5kdc[7300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:20:05.322265 7385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:05.312396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43256 (local address 127.4.202.190:46843)
0926 06:20:05.312602 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:05.312609 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:05.312642 (+ 33us) server_negotiation.cc:407] Connection header received
0926 06:20:05.312732 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:05.312737 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:05.312782 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:05.312892 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:05.313713 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:05.314788 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:05.315850 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:05.316473 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:05.316572 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:05.319200 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:05.319204 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:05.319204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:05.319216 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:05.320648 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:05.321003 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:05.321006 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:05.321006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:05.321107 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:05.321318 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:05.321319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:05.321320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:05.321559 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:05.321734 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:05.321796 (+ 62us) server_negotiation.cc:299] Negotiation successful
0926 06:20:05.321945 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":108,"threads_started":1}
I20260926 06:20:05.327986 7394 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:20:05.328476 7394 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:20:05.332358 7394 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c56fbeb893a4bf185d475fad83f78cb: Generated new TSK 0
I20260926 06:20:05.332523 7394 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.202.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
I20260926 06:20:05.363710 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.129:0
--local_ip_for_outbound_sockets=127.4.202.129
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:46843
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:35221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:05.477340 7407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:05.477602 7407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:05.477635 7407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:05.479425 7407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:05.479490 7407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:05.479526 7407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.129
I20260926 06:20:05.481663 7407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:35221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:46843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.129
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:05.481886 7407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:05.482112 7407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:05.487102 7415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:05.487078 7412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:05.487077 7413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:05.487529 7407 server_base.cc:1034] running on GCE node
I20260926 06:20:05.487741 7407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:05.487982 7407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:05.489152 7407 hybrid_clock.cc:648] HybridClock initialized: now 1790403605489134 us; error 33 us; skew 500 ppm
Sep 26 06:20:05 dist-test-slave-pd0t krb5kdc[7300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403605, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:05.491593 7407 init.cc:377] Logged in from keytab as kudu/127.4.202.129@KRBTEST.COM (short username kudu)
I20260926 06:20:05.491959 7407 webserver.cc:460] Webserver started at http://127.4.202.129:43625/ using document root <none> and password file <none>
I20260926 06:20:05.492130 7407 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:05.492169 7407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:05.492252 7407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:05.493235 7407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/data/instance:
uuid: "47e07103fc4a4144b1e3d929e2b3dd68"
format_stamp: "Formatted at 2026-09-26 06:20:05 on dist-test-slave-pd0t"
server_key: "33942e3eecbae19b85ac83a67dded0ce"
server_key_iv: "593b481f1d9fd836edcb3a499bf1e185"
server_key_version: "encryptionkey@0"
I20260926 06:20:05.493530 7407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance:
uuid: "47e07103fc4a4144b1e3d929e2b3dd68"
format_stamp: "Formatted at 2026-09-26 06:20:05 on dist-test-slave-pd0t"
server_key: "33942e3eecbae19b85ac83a67dded0ce"
server_key_iv: "593b481f1d9fd836edcb3a499bf1e185"
server_key_version: "encryptionkey@0"
I20260926 06:20:05.494813 7407 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:20:05.495498 7422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:05.495651 7407 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:20:05.495734 7407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/wal
uuid: "47e07103fc4a4144b1e3d929e2b3dd68"
format_stamp: "Formatted at 2026-09-26 06:20:05 on dist-test-slave-pd0t"
server_key: "33942e3eecbae19b85ac83a67dded0ce"
server_key_iv: "593b481f1d9fd836edcb3a499bf1e185"
server_key_version: "encryptionkey@0"
I20260926 06:20:05.495786 7407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:05.517939 7407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:05.519793 7407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:05.519901 7407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:05.520080 7407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:20:05.520323 7407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:20:05.520351 7407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:05.520373 7407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:20:05.520386 7407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:05.525882 7407 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.129:40137
I20260926 06:20:05.525965 7535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.129:40137 every 8 connection(s)
I20260926 06:20:05.526278 7407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
I20260926 06:20:05.531989 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 7407
I20260926 06:20:05.532088 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance
I20260926 06:20:05.532253 4906 external_mini_cluster.cc:1428] Setting key 19be0414c690cbb1af86a98c57f4fae4
Sep 26 06:20:05 dist-test-slave-pd0t krb5kdc[7300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403605, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:20:05.536085 7538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:05.526949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:46843 (local address 127.4.202.129:38523)
0926 06:20:05.527181 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:05.527227 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:05.527764 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:05.528035 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:05.528039 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:05.528629 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:05.528808 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:05.528812 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:05.529556 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:05.529557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:05.530357 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:05.530360 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:05.530834 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:05.530835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:05.530962 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:05.531343 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:05.531350 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:05.532916 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:05.534828 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:05.534833 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:05.534835 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:05.535051 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:05.535249 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:05.535250 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:05.535250 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:05.535339 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:05.535613 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:05.535617 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:05.535718 (+ 101us) client_negotiation.cc:769] Sending connection context
0926 06:20:05.535805 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 06:20:05.535929 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":95,"threads_started":1}
I20260926 06:20:05.536088 7385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:05.526979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:38523 (local address 127.4.202.190:46843)
0926 06:20:05.527124 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:05.527126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:05.527326 (+ 200us) server_negotiation.cc:407] Connection header received
0926 06:20:05.527870 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:05.527871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:05.527886 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:05.527938 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:05.528912 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:05.529403 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:05.530459 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:05.530738 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:05.530824 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:05.533023 (+ 2199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:05.533024 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:05.533025 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:05.533048 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:05.534752 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:05.535126 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:05.535127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:05.535127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:05.535190 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:05.535398 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:05.535399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:05.535400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:05.535548 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:05.535617 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:05.535849 (+ 232us) server_negotiation.cc:299] Negotiation successful
0926 06:20:05.535954 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 06:20:05.536617 7536 heartbeater.cc:347] Connected to a master server at 127.4.202.190:46843
I20260926 06:20:05.536720 7536 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:05.536911 7536 heartbeater.cc:511] Master 127.4.202.190:46843 requested a full tablet report, sending...
I20260926 06:20:05.537366 7345 ts_manager.cc:194] Registered new tserver with Master: 47e07103fc4a4144b1e3d929e2b3dd68 (127.4.202.129:40137)
I20260926 06:20:05.537940 7345 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.129@KRBTEST.COM'} at 127.4.202.129:38523
WARNING: no policy specified for kudu/127.4.202.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
I20260926 06:20:05.556843 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.130:0
--local_ip_for_outbound_sockets=127.4.202.130
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:46843
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:35221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:05.635449 7541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:05.635641 7541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:05.635677 7541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:05.636678 7541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:05.636708 7541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:05.636731 7541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.130
I20260926 06:20:05.637980 7541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:35221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:46843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.130
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:05.638149 7541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:05.638303 7541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:05.643013 7549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:05.643121 7541 server_base.cc:1034] running on GCE node
W20260926 06:20:05.642984 7546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:05.642997 7547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:05.643489 7541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:05.643720 7541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:05.644883 7541 hybrid_clock.cc:648] HybridClock initialized: now 1790403605644873 us; error 26 us; skew 500 ppm
Sep 26 06:20:05 dist-test-slave-pd0t krb5kdc[7300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403605, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:05.647368 7541 init.cc:377] Logged in from keytab as kudu/127.4.202.130@KRBTEST.COM (short username kudu)
I20260926 06:20:05.647823 7541 webserver.cc:460] Webserver started at http://127.4.202.130:40531/ using document root <none> and password file <none>
I20260926 06:20:05.648077 7541 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:05.648146 7541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:05.648307 7541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:05.649240 7541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/data/instance:
uuid: "a6e312f78d9f4584b69dc6cc9491d6bc"
format_stamp: "Formatted at 2026-09-26 06:20:05 on dist-test-slave-pd0t"
server_key: "0303f785b276826a528221c018dd2689"
server_key_iv: "4901d53cffa7fc3b4b2e88765535a2e8"
server_key_version: "encryptionkey@0"
I20260926 06:20:05.649578 7541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance:
uuid: "a6e312f78d9f4584b69dc6cc9491d6bc"
format_stamp: "Formatted at 2026-09-26 06:20:05 on dist-test-slave-pd0t"
server_key: "0303f785b276826a528221c018dd2689"
server_key_iv: "4901d53cffa7fc3b4b2e88765535a2e8"
server_key_version: "encryptionkey@0"
I20260926 06:20:05.650864 7541 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 06:20:05.651533 7556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:05.651693 7541 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:20:05.651782 7541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/wal
uuid: "a6e312f78d9f4584b69dc6cc9491d6bc"
format_stamp: "Formatted at 2026-09-26 06:20:05 on dist-test-slave-pd0t"
server_key: "0303f785b276826a528221c018dd2689"
server_key_iv: "4901d53cffa7fc3b4b2e88765535a2e8"
server_key_version: "encryptionkey@0"
I20260926 06:20:05.651858 7541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:05.684425 7541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:05.686722 7541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:05.686885 7541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:05.687112 7541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:20:05.687423 7541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:20:05.687477 7541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:05.687508 7541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:20:05.687537 7541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:05.693382 7541 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.130:33053
I20260926 06:20:05.693435 7669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.130:33053 every 8 connection(s)
I20260926 06:20:05.693751 7541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
Sep 26 06:20:05 dist-test-slave-pd0t krb5kdc[7300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403605, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:20:05.701099 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 7541
I20260926 06:20:05.701174 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance
I20260926 06:20:05.701357 4906 external_mini_cluster.cc:1428] Setting key 2929ddaf985ca84078a80bea32f70ca3
I20260926 06:20:05.702940 7672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:05.694340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:46843 (local address 127.4.202.130:33143)
0926 06:20:05.694493 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:05.694530 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:05.694995 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:05.695213 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:05.695216 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:05.695734 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:05.695910 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:05.695914 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:05.696566 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:05.696567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:05.697363 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:05.697365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:05.697671 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:05.697672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:05.697767 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:05.698170 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:05.698178 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:05.699688 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:05.701311 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:05.701318 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:05.701320 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:05.701651 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:05.701872 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:05.701873 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:05.701873 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:05.701999 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:05.702376 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:05.702383 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:05.702519 (+ 136us) client_negotiation.cc:769] Sending connection context
0926 06:20:05.702670 (+ 151us) client_negotiation.cc:240] Negotiation successful
0926 06:20:05.702794 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":91,"threads_started":1}
I20260926 06:20:05.702934 7385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:05.694378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:33143 (local address 127.4.202.190:46843)
0926 06:20:05.694446 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:05.694448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:05.694613 (+ 165us) server_negotiation.cc:407] Connection header received
0926 06:20:05.695070 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:05.695070 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:05.695080 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:05.695158 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:05.695996 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:05.696487 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:05.697435 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:05.697604 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:05.697664 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:05.699766 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:05.699767 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:05.699768 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:05.699774 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:05.701218 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:05.701742 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:05.701744 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:05.701744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:05.701809 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:05.702079 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:05.702080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:05.702081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:05.702241 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:05.702307 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:05.702704 (+ 397us) server_negotiation.cc:299] Negotiation successful
0926 06:20:05.702796 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 06:20:05.703670 7670 heartbeater.cc:347] Connected to a master server at 127.4.202.190:46843
I20260926 06:20:05.703769 7670 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:05.703943 7670 heartbeater.cc:511] Master 127.4.202.190:46843 requested a full tablet report, sending...
I20260926 06:20:05.704268 7345 ts_manager.cc:194] Registered new tserver with Master: a6e312f78d9f4584b69dc6cc9491d6bc (127.4.202.130:33053)
I20260926 06:20:05.704636 7345 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.130@KRBTEST.COM'} at 127.4.202.130:33143
WARNING: no policy specified for kudu/127.4.202.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
I20260926 06:20:05.726176 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.131:0
--local_ip_for_outbound_sockets=127.4.202.131
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:46843
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:35221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:05.802316 7675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:05.802523 7675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:05.802561 7675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:05.803675 7675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:05.803738 7675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:05.803781 7675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.131
I20260926 06:20:05.805209 7675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:35221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:46843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.131
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:05.805449 7675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:05.805662 7675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:05.810432 7681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:05.810464 7680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:05.810415 7675 server_base.cc:1034] running on GCE node
W20260926 06:20:05.810415 7683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:05.810969 7675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:05.811199 7675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:05.812376 7675 hybrid_clock.cc:648] HybridClock initialized: now 1790403605812358 us; error 30 us; skew 500 ppm
Sep 26 06:20:05 dist-test-slave-pd0t krb5kdc[7300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403605, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:05.814812 7675 init.cc:377] Logged in from keytab as kudu/127.4.202.131@KRBTEST.COM (short username kudu)
I20260926 06:20:05.815208 7675 webserver.cc:460] Webserver started at http://127.4.202.131:40219/ using document root <none> and password file <none>
I20260926 06:20:05.815477 7675 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:05.815578 7675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:05.815706 7675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:05.816804 7675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/data/instance:
uuid: "7690f069fca64cf5bae4be66b29e9c26"
format_stamp: "Formatted at 2026-09-26 06:20:05 on dist-test-slave-pd0t"
server_key: "15d27af41afe145bd2da5bdbee4efb6e"
server_key_iv: "3a91e196f58820178092b8ccc9cf30b0"
server_key_version: "encryptionkey@0"
I20260926 06:20:05.817179 7675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance:
uuid: "7690f069fca64cf5bae4be66b29e9c26"
format_stamp: "Formatted at 2026-09-26 06:20:05 on dist-test-slave-pd0t"
server_key: "15d27af41afe145bd2da5bdbee4efb6e"
server_key_iv: "3a91e196f58820178092b8ccc9cf30b0"
server_key_version: "encryptionkey@0"
I20260926 06:20:05.818475 7675 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:20:05.819286 7690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:05.819501 7675 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:20:05.819564 7675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/wal
uuid: "7690f069fca64cf5bae4be66b29e9c26"
format_stamp: "Formatted at 2026-09-26 06:20:05 on dist-test-slave-pd0t"
server_key: "15d27af41afe145bd2da5bdbee4efb6e"
server_key_iv: "3a91e196f58820178092b8ccc9cf30b0"
server_key_version: "encryptionkey@0"
I20260926 06:20:05.819649 7675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:05.829802 7675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:05.831789 7675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:05.831924 7675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:05.832154 7675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:20:05.832541 7675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:20:05.832587 7675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:05.832617 7675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:20:05.832629 7675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:05.838315 7675 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.131:33791
I20260926 06:20:05.838354 7803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.131:33791 every 8 connection(s)
I20260926 06:20:05.838711 7675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
I20260926 06:20:05.840200 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 7675
I20260926 06:20:05.840269 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance
I20260926 06:20:05.840420 4906 external_mini_cluster.cc:1428] Setting key 3ff850de30d43e71f8f071f1c464d144
Sep 26 06:20:05 dist-test-slave-pd0t krb5kdc[7300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403605, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:20:05.847844 7385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:05.839363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:58291 (local address 127.4.202.190:46843)
0926 06:20:05.839404 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:05.839406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:05.839608 (+ 202us) server_negotiation.cc:407] Connection header received
0926 06:20:05.840075 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:05.840076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:05.840090 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:05.840134 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:05.841153 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:05.841632 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:05.842654 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:05.842847 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:05.842944 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:05.845051 (+ 2107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:05.845052 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:05.845052 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:05.845060 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:05.846416 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:05.846876 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:05.846877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:05.846878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:05.846948 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:05.847202 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:05.847203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:05.847204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:05.847314 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:05.847390 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:05.847678 (+ 288us) server_negotiation.cc:299] Negotiation successful
0926 06:20:05.847755 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 06:20:05.848028 7806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:05.839321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:46843 (local address 127.4.202.131:58291)
0926 06:20:05.839476 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:05.839513 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:05.839969 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:05.840197 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:05.840201 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:05.840839 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:05.841042 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:05.841047 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:05.841729 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:05.841730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:05.842528 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:05.842531 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:05.842961 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:05.842962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:05.843097 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:05.843578 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:05.843586 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:05.844944 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:05.846518 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:05.846524 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:05.846526 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:05.846786 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:05.847017 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:05.847018 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:05.847018 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:05.847125 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:05.847380 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:05.847386 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:05.847546 (+ 160us) client_negotiation.cc:769] Sending connection context
0926 06:20:05.847656 (+ 110us) client_negotiation.cc:240] Negotiation successful
0926 06:20:05.847803 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":90,"threads_started":1}
I20260926 06:20:05.848436 7804 heartbeater.cc:347] Connected to a master server at 127.4.202.190:46843
I20260926 06:20:05.848570 7804 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:05.848739 7804 heartbeater.cc:511] Master 127.4.202.190:46843 requested a full tablet report, sending...
I20260926 06:20:05.849136 7345 ts_manager.cc:194] Registered new tserver with Master: 7690f069fca64cf5bae4be66b29e9c26 (127.4.202.131:33791)
I20260926 06:20:05.849457 7345 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.131@KRBTEST.COM'} at 127.4.202.131:58291
I20260926 06:20:05.851848 4906 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 06:20:05.862474 7385 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:20:05.861655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43270 (local address 127.4.202.190:46843)
0926 06:20:05.861706 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:05.861708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:05.861746 (+ 38us) server_negotiation.cc:407] Connection header received
0926 06:20:05.861838 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:05.861844 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:05.861852 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:05.861913 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:05.862429 (+ 516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43270: BlockingRecv error: recv got EOF from 127.0.0.1:43270 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 06:20:05.863466 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 7407
I20260926 06:20:05.867957 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 7541
I20260926 06:20:05.871876 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 7675
I20260926 06:20:05.875756 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 7313
2026-09-26T06:20:05Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1827 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:20:05 dist-test-slave-pd0t krb5kdc[7816](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:20:05 dist-test-slave-pd0t krb5kdc[7816](info): set up 2 sockets
Sep 26 06:20:05 dist-test-slave-pd0t krb5kdc[7816](info): commencing operation
krb5kdc: starting...
W20260926 06:20:06.890251 4906 mini_kdc.cc:121] Time spent starting KDC: real 0.995s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:20:06 dist-test-slave-pd0t krb5kdc[7816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:20:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:20:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.202.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
I20260926 06:20:07.001773 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:34759
--webserver_interface=127.4.202.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:42761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:34759
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:07.078965 7829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:07.079210 7829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:07.079244 7829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:07.080397 7829 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:20:07.080468 7829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:07.080493 7829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:07.080526 7829 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:20:07.080552 7829 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:20:07.082016 7829 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:42761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:34759
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:34759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:07.082273 7829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:07.082491 7829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:07.087332 7835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:07.087276 7834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:07.087276 7837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:07.087519 7829 server_base.cc:1034] running on GCE node
I20260926 06:20:07.087795 7829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:07.088040 7829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:07.089221 7829 hybrid_clock.cc:648] HybridClock initialized: now 1790403607089203 us; error 38 us; skew 500 ppm
Sep 26 06:20:07 dist-test-slave-pd0t krb5kdc[7816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403607, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:07.091722 7829 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:20:07.092149 7829 webserver.cc:460] Webserver started at http://127.4.202.190:37763/ using document root <none> and password file <none>
I20260926 06:20:07.092391 7829 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:07.092461 7829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:07.092607 7829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:07.093605 7829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "ce7d6d34311445ba8ceed2375e2bb8d8"
format_stamp: "Formatted at 2026-09-26 06:20:07 on dist-test-slave-pd0t"
server_key: "eb49283fc0ee33b76bdfd5c42a940060"
server_key_iv: "82e9d620e4f4f9d1d0ae3a3f65fc50b5"
server_key_version: "encryptionkey@0"
I20260926 06:20:07.093964 7829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "ce7d6d34311445ba8ceed2375e2bb8d8"
format_stamp: "Formatted at 2026-09-26 06:20:07 on dist-test-slave-pd0t"
server_key: "eb49283fc0ee33b76bdfd5c42a940060"
server_key_iv: "82e9d620e4f4f9d1d0ae3a3f65fc50b5"
server_key_version: "encryptionkey@0"
I20260926 06:20:07.095431 7829 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:20:07.096216 7844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:07.096385 7829 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:20:07.096485 7829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "ce7d6d34311445ba8ceed2375e2bb8d8"
format_stamp: "Formatted at 2026-09-26 06:20:07 on dist-test-slave-pd0t"
server_key: "eb49283fc0ee33b76bdfd5c42a940060"
server_key_iv: "82e9d620e4f4f9d1d0ae3a3f65fc50b5"
server_key_version: "encryptionkey@0"
I20260926 06:20:07.096568 7829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:07.120268 7829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:07.122704 7829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:07.122862 7829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:07.126734 7829 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:34759
I20260926 06:20:07.126770 7896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:34759 every 8 connection(s)
I20260926 06:20:07.127140 7829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:20:07.128010 7897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:07.130151 7897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce7d6d34311445ba8ceed2375e2bb8d8: Bootstrap starting.
I20260926 06:20:07.130746 7897 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce7d6d34311445ba8ceed2375e2bb8d8: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:07.130981 7897 log.cc:824] T 00000000000000000000000000000000 P ce7d6d34311445ba8ceed2375e2bb8d8: Log is configured to *not* fsync() on all Append() calls
I20260926 06:20:07.131568 7897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce7d6d34311445ba8ceed2375e2bb8d8: No bootstrap required, opened a new log
I20260926 06:20:07.133016 7897 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce7d6d34311445ba8ceed2375e2bb8d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce7d6d34311445ba8ceed2375e2bb8d8" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34759 } }
I20260926 06:20:07.133141 7897 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce7d6d34311445ba8ceed2375e2bb8d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:07.133163 7897 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce7d6d34311445ba8ceed2375e2bb8d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce7d6d34311445ba8ceed2375e2bb8d8, State: Initialized, Role: FOLLOWER
I20260926 06:20:07.133288 7897 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce7d6d34311445ba8ceed2375e2bb8d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce7d6d34311445ba8ceed2375e2bb8d8" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34759 } }
I20260926 06:20:07.133365 7897 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce7d6d34311445ba8ceed2375e2bb8d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:20:07.133387 7897 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce7d6d34311445ba8ceed2375e2bb8d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:20:07.133416 7897 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce7d6d34311445ba8ceed2375e2bb8d8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:07.134148 7897 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce7d6d34311445ba8ceed2375e2bb8d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce7d6d34311445ba8ceed2375e2bb8d8" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34759 } }
I20260926 06:20:07.134230 7897 leader_election.cc:302] T 00000000000000000000000000000000 P ce7d6d34311445ba8ceed2375e2bb8d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce7d6d34311445ba8ceed2375e2bb8d8; no voters:
I20260926 06:20:07.134366 7897 leader_election.cc:288] T 00000000000000000000000000000000 P ce7d6d34311445ba8ceed2375e2bb8d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:20:07.134414 7900 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce7d6d34311445ba8ceed2375e2bb8d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:20:07.134533 7900 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce7d6d34311445ba8ceed2375e2bb8d8 [term 1 LEADER]: Becoming Leader. State: Replica: ce7d6d34311445ba8ceed2375e2bb8d8, State: Running, Role: LEADER
I20260926 06:20:07.134645 7897 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce7d6d34311445ba8ceed2375e2bb8d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:20:07.134631 7900 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce7d6d34311445ba8ceed2375e2bb8d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce7d6d34311445ba8ceed2375e2bb8d8" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34759 } }
I20260926 06:20:07.135149 7901 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce7d6d34311445ba8ceed2375e2bb8d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce7d6d34311445ba8ceed2375e2bb8d8. Latest consensus state: current_term: 1 leader_uuid: "ce7d6d34311445ba8ceed2375e2bb8d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce7d6d34311445ba8ceed2375e2bb8d8" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34759 } } }
I20260926 06:20:07.135164 7902 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce7d6d34311445ba8ceed2375e2bb8d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce7d6d34311445ba8ceed2375e2bb8d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce7d6d34311445ba8ceed2375e2bb8d8" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34759 } } }
I20260926 06:20:07.135368 7902 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce7d6d34311445ba8ceed2375e2bb8d8 [sys.catalog]: This master's current role is: LEADER
I20260926 06:20:07.135356 7901 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce7d6d34311445ba8ceed2375e2bb8d8 [sys.catalog]: This master's current role is: LEADER
I20260926 06:20:07.135600 7909 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:20:07.136148 7909 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:20:07.136411 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 7829
I20260926 06:20:07.136511 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:20:07.136675 4906 external_mini_cluster.cc:1428] Setting key c1630215eac4199d41f5ffee00be2a4a
I20260926 06:20:07.137790 7909 catalog_manager.cc:1347] Generated new cluster ID: 8d170fb235ae46faa1f5c271a1cf223f
I20260926 06:20:07.137840 7909 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:20:07 dist-test-slave-pd0t krb5kdc[7816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:20:07.147998 7919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:07.137274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33220 (local address 127.4.202.190:34759)
0926 06:20:07.137430 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:07.137439 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:07.137471 (+ 32us) server_negotiation.cc:407] Connection header received
0926 06:20:07.137580 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:07.137586 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:07.137638 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:07.137787 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:07.138639 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:07.139962 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:07.141005 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:07.141598 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:07.141682 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:07.144530 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:07.144534 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:07.144534 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:07.144545 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:07.146363 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:07.146804 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:07.146807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:07.146807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:07.146909 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:07.147124 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:07.147125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:07.147125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:07.147399 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:07.147548 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:07.147645 (+ 97us) server_negotiation.cc:299] Negotiation successful
0926 06:20:07.147769 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":93,"threads_started":1}
I20260926 06:20:07.148629 7909 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:20:07.149062 4906 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:20:07.149307 7909 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:20:07.155112 7909 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce7d6d34311445ba8ceed2375e2bb8d8: Generated new TSK 0
I20260926 06:20:07.155313 7909 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:20:07.163167 7919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:07.159652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33236 (local address 127.4.202.190:34759)
0926 06:20:07.159712 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:07.159714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:07.159849 (+ 135us) server_negotiation.cc:407] Connection header received
0926 06:20:07.159914 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:07.159915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:07.159933 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:07.159976 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:07.160358 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:07.160816 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:07.161820 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:07.162007 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:07.162054 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:07.162728 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:07.162729 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:07.162729 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:20:07.162889 (+ 160us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:07.162903 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:20:07.162908 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:07.162965 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:07.163051 (+ 86us) server_negotiation.cc:299] Negotiation successful
0926 06:20:07.163072 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 06:20:07.165561 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 7829
2026-09-26T06:20:07Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1288 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:20:07 dist-test-slave-pd0t krb5kdc[7930](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:20:07 dist-test-slave-pd0t krb5kdc[7930](info): set up 2 sockets
Sep 26 06:20:07 dist-test-slave-pd0t krb5kdc[7930](info): commencing operation
krb5kdc: starting...
W20260926 06:20:08.182276 4906 mini_kdc.cc:121] Time spent starting KDC: real 0.999s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:20:08 dist-test-slave-pd0t krb5kdc[7930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:20:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:20:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.202.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
I20260926 06:20:08.291333 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:41167
--webserver_interface=127.4.202.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:41627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:41167
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:08.368854 7943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:08.369091 7943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:08.369127 7943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:08.370298 7943 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:20:08.370368 7943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:08.370393 7943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:08.370414 7943 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:20:08.370442 7943 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:20:08.371870 7943 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:41627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:41167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:41167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:08.372155 7943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:08.372390 7943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:08.376955 7951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:08.376991 7949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:08.377169 7943 server_base.cc:1034] running on GCE node
W20260926 06:20:08.376989 7948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:08.377470 7943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:08.377736 7943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:08.378893 7943 hybrid_clock.cc:648] HybridClock initialized: now 1790403608378892 us; error 40 us; skew 500 ppm
Sep 26 06:20:08 dist-test-slave-pd0t krb5kdc[7930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403608, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:08.381143 7943 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:20:08.381516 7943 webserver.cc:460] Webserver started at http://127.4.202.190:39445/ using document root <none> and password file <none>
I20260926 06:20:08.381682 7943 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:08.381721 7943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:08.381804 7943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:08.382826 7943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "860330aa48534d4c80ec5d40a5a11420"
format_stamp: "Formatted at 2026-09-26 06:20:08 on dist-test-slave-pd0t"
server_key: "ce40f352e30636b96e247066560b83e1"
server_key_iv: "59a6cf5b7d87e459adbd07e14fd3b26e"
server_key_version: "encryptionkey@0"
I20260926 06:20:08.383175 7943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "860330aa48534d4c80ec5d40a5a11420"
format_stamp: "Formatted at 2026-09-26 06:20:08 on dist-test-slave-pd0t"
server_key: "ce40f352e30636b96e247066560b83e1"
server_key_iv: "59a6cf5b7d87e459adbd07e14fd3b26e"
server_key_version: "encryptionkey@0"
I20260926 06:20:08.384450 7943 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:20:08.385149 7958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:08.385342 7943 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:20:08.385442 7943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "860330aa48534d4c80ec5d40a5a11420"
format_stamp: "Formatted at 2026-09-26 06:20:08 on dist-test-slave-pd0t"
server_key: "ce40f352e30636b96e247066560b83e1"
server_key_iv: "59a6cf5b7d87e459adbd07e14fd3b26e"
server_key_version: "encryptionkey@0"
I20260926 06:20:08.385510 7943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:08.400064 7943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:08.401825 7943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:08.401975 7943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:08.405858 7943 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:41167
I20260926 06:20:08.405889 8010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:41167 every 8 connection(s)
I20260926 06:20:08.406241 7943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:20:08.407191 8011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:08.409518 8011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 860330aa48534d4c80ec5d40a5a11420: Bootstrap starting.
I20260926 06:20:08.410094 8011 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 860330aa48534d4c80ec5d40a5a11420: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:08.410362 8011 log.cc:824] T 00000000000000000000000000000000 P 860330aa48534d4c80ec5d40a5a11420: Log is configured to *not* fsync() on all Append() calls
I20260926 06:20:08.411088 8011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 860330aa48534d4c80ec5d40a5a11420: No bootstrap required, opened a new log
I20260926 06:20:08.412642 8011 raft_consensus.cc:348] T 00000000000000000000000000000000 P 860330aa48534d4c80ec5d40a5a11420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "860330aa48534d4c80ec5d40a5a11420" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 41167 } }
I20260926 06:20:08.412770 8011 raft_consensus.cc:374] T 00000000000000000000000000000000 P 860330aa48534d4c80ec5d40a5a11420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:08.412792 8011 raft_consensus.cc:729] T 00000000000000000000000000000000 P 860330aa48534d4c80ec5d40a5a11420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 860330aa48534d4c80ec5d40a5a11420, State: Initialized, Role: FOLLOWER
I20260926 06:20:08.412969 8011 consensus_queue.cc:260] T 00000000000000000000000000000000 P 860330aa48534d4c80ec5d40a5a11420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "860330aa48534d4c80ec5d40a5a11420" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 41167 } }
I20260926 06:20:08.413050 8011 raft_consensus.cc:388] T 00000000000000000000000000000000 P 860330aa48534d4c80ec5d40a5a11420 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:20:08.413100 8011 raft_consensus.cc:482] T 00000000000000000000000000000000 P 860330aa48534d4c80ec5d40a5a11420 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:20:08.413147 8011 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 860330aa48534d4c80ec5d40a5a11420 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:08.413892 8011 raft_consensus.cc:504] T 00000000000000000000000000000000 P 860330aa48534d4c80ec5d40a5a11420 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "860330aa48534d4c80ec5d40a5a11420" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 41167 } }
I20260926 06:20:08.414000 8011 leader_election.cc:302] T 00000000000000000000000000000000 P 860330aa48534d4c80ec5d40a5a11420 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 860330aa48534d4c80ec5d40a5a11420; no voters:
I20260926 06:20:08.414160 8011 leader_election.cc:288] T 00000000000000000000000000000000 P 860330aa48534d4c80ec5d40a5a11420 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:20:08.414188 8014 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 860330aa48534d4c80ec5d40a5a11420 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:20:08.414417 8011 sys_catalog.cc:564] T 00000000000000000000000000000000 P 860330aa48534d4c80ec5d40a5a11420 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:20:08.414426 8014 raft_consensus.cc:686] T 00000000000000000000000000000000 P 860330aa48534d4c80ec5d40a5a11420 [term 1 LEADER]: Becoming Leader. State: Replica: 860330aa48534d4c80ec5d40a5a11420, State: Running, Role: LEADER
I20260926 06:20:08.414624 8014 consensus_queue.cc:237] T 00000000000000000000000000000000 P 860330aa48534d4c80ec5d40a5a11420 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "860330aa48534d4c80ec5d40a5a11420" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 41167 } }
I20260926 06:20:08.414990 8015 sys_catalog.cc:455] T 00000000000000000000000000000000 P 860330aa48534d4c80ec5d40a5a11420 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "860330aa48534d4c80ec5d40a5a11420" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "860330aa48534d4c80ec5d40a5a11420" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 41167 } } }
I20260926 06:20:08.415000 8016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 860330aa48534d4c80ec5d40a5a11420 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 860330aa48534d4c80ec5d40a5a11420. Latest consensus state: current_term: 1 leader_uuid: "860330aa48534d4c80ec5d40a5a11420" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "860330aa48534d4c80ec5d40a5a11420" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 41167 } } }
I20260926 06:20:08.415164 8015 sys_catalog.cc:458] T 00000000000000000000000000000000 P 860330aa48534d4c80ec5d40a5a11420 [sys.catalog]: This master's current role is: LEADER
I20260926 06:20:08.415174 8016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 860330aa48534d4c80ec5d40a5a11420 [sys.catalog]: This master's current role is: LEADER
I20260926 06:20:08.415413 8023 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:20:08.415555 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 7943
I20260926 06:20:08.415657 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:20:08.415863 4906 external_mini_cluster.cc:1428] Setting key e46ad978c92c1c93440e5a4c7c21a9cb
I20260926 06:20:08.416255 8023 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:20:08.418027 8023 catalog_manager.cc:1347] Generated new cluster ID: 63ed15174ebf4167bc17c82aec5031b8
I20260926 06:20:08.418084 8023 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:20:08 dist-test-slave-pd0t krb5kdc[7930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:20:08.427387 8023 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:20:08.427475 8033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:08.416467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33448 (local address 127.4.202.190:41167)
0926 06:20:08.416644 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:08.416653 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:08.416684 (+ 31us) server_negotiation.cc:407] Connection header received
0926 06:20:08.416833 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:08.416840 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:08.416899 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:08.417040 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:08.417991 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:08.419424 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:08.420496 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:08.421056 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:08.421145 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:08.423824 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:08.423828 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:08.423828 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:08.423840 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:08.425809 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:08.426233 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:08.426236 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:08.426236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:08.426353 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:08.426572 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:08.426573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:08.426573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:08.426859 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:08.427019 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:08.427132 (+ 113us) server_negotiation.cc:299] Negotiation successful
0926 06:20:08.427262 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":109,"threads_started":1}
I20260926 06:20:08.428051 8023 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:20:08.435546 8023 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 860330aa48534d4c80ec5d40a5a11420: Generated new TSK 0
I20260926 06:20:08.435710 8023 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.202.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
I20260926 06:20:08.459858 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.129:0
--local_ip_for_outbound_sockets=127.4.202.129
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:41167
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:41627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:08.539247 8037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:08.539433 8037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:08.539455 8037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:08.540535 8037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:08.540575 8037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:08.540599 8037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.129
I20260926 06:20:08.542022 8037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:41627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:41167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.129
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:08.542253 8037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:08.542470 8037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:08.547341 8042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:08.547353 8045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:08.547353 8043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:08.547677 8037 server_base.cc:1034] running on GCE node
I20260926 06:20:08.547917 8037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:08.548135 8037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:08.549299 8037 hybrid_clock.cc:648] HybridClock initialized: now 1790403608549292 us; error 37 us; skew 500 ppm
Sep 26 06:20:08 dist-test-slave-pd0t krb5kdc[7930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403608, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:08.551888 8037 init.cc:377] Logged in from keytab as kudu/127.4.202.129@KRBTEST.COM (short username kudu)
I20260926 06:20:08.552448 8037 webserver.cc:460] Webserver started at http://127.4.202.129:33919/ using document root <none> and password file <none>
I20260926 06:20:08.552724 8037 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:08.552804 8037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:08.552937 8037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:08.554484 8037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/data/instance:
uuid: "115aa30def3b496b9c9b98c0db8f91ac"
format_stamp: "Formatted at 2026-09-26 06:20:08 on dist-test-slave-pd0t"
server_key: "7040e907ce38f791207cc1bd8b419d9d"
server_key_iv: "9a2dc2676622585095e4e99231fdee93"
server_key_version: "encryptionkey@0"
I20260926 06:20:08.554992 8037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance:
uuid: "115aa30def3b496b9c9b98c0db8f91ac"
format_stamp: "Formatted at 2026-09-26 06:20:08 on dist-test-slave-pd0t"
server_key: "7040e907ce38f791207cc1bd8b419d9d"
server_key_iv: "9a2dc2676622585095e4e99231fdee93"
server_key_version: "encryptionkey@0"
I20260926 06:20:08.556743 8037 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260926 06:20:08.557618 8052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:08.557791 8037 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:20:08.557901 8037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/wal
uuid: "115aa30def3b496b9c9b98c0db8f91ac"
format_stamp: "Formatted at 2026-09-26 06:20:08 on dist-test-slave-pd0t"
server_key: "7040e907ce38f791207cc1bd8b419d9d"
server_key_iv: "9a2dc2676622585095e4e99231fdee93"
server_key_version: "encryptionkey@0"
I20260926 06:20:08.557988 8037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:08.575672 8037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:08.577863 8037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:08.578032 8037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:08.578238 8037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:20:08.578521 8037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:20:08.578573 8037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:08.578632 8037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:20:08.578661 8037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:08.584304 8037 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.129:44689
I20260926 06:20:08.584347 8165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.129:44689 every 8 connection(s)
I20260926 06:20:08.584618 8037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
Sep 26 06:20:08 dist-test-slave-pd0t krb5kdc[7930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403608, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:20:08.592661 8033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:08.585264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:43101 (local address 127.4.202.190:41167)
0926 06:20:08.585303 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:08.585305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:08.585470 (+ 165us) server_negotiation.cc:407] Connection header received
0926 06:20:08.585871 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:08.585872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:08.585882 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:08.585961 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:08.586818 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:08.587264 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:08.588177 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:08.588373 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:08.588472 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:08.590383 (+ 1911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:08.590385 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:08.590385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:08.590392 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:08.591559 (+ 1167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:08.591884 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:08.591885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:08.591885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:08.591942 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:08.592134 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:08.592135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:08.592135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:08.592221 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:08.592267 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:08.592540 (+ 273us) server_negotiation.cc:299] Negotiation successful
0926 06:20:08.592599 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 06:20:08.592808 8168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:08.585219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41167 (local address 127.4.202.129:43101)
0926 06:20:08.585365 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:08.585407 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:08.585800 (+ 393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:08.586006 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:08.586009 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:08.586530 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:08.586753 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:08.586758 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:08.587334 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:08.587334 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:08.588112 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:08.588114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:08.588479 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:08.588480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:08.588572 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:08.589034 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:08.589041 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:08.590307 (+ 1266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:08.591630 (+ 1323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:08.591635 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:08.591637 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:08.591824 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:08.591998 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:08.591998 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:08.591998 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:08.592084 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:08.592272 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:08.592276 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:08.592433 (+ 157us) client_negotiation.cc:769] Sending connection context
0926 06:20:08.592510 (+ 77us) client_negotiation.cc:240] Negotiation successful
0926 06:20:08.592613 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":88,"threads_started":1}
I20260926 06:20:08.593323 8166 heartbeater.cc:347] Connected to a master server at 127.4.202.190:41167
I20260926 06:20:08.593427 8166 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:08.593559 8166 heartbeater.cc:511] Master 127.4.202.190:41167 requested a full tablet report, sending...
I20260926 06:20:08.594058 7975 ts_manager.cc:194] Registered new tserver with Master: 115aa30def3b496b9c9b98c0db8f91ac (127.4.202.129:44689)
I20260926 06:20:08.594120 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 8037
I20260926 06:20:08.594199 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance
I20260926 06:20:08.594389 4906 external_mini_cluster.cc:1428] Setting key 5a6ac32de412ddbb0a56eb97a16bb7b7
I20260926 06:20:08.594887 7975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.129@KRBTEST.COM'} at 127.4.202.129:43101
WARNING: no policy specified for kudu/127.4.202.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
I20260926 06:20:08.618054 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.130:0
--local_ip_for_outbound_sockets=127.4.202.130
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:41167
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:41627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:08.695611 8171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:08.695786 8171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:08.695807 8171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:08.696820 8171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:08.696861 8171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:08.696885 8171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.130
I20260926 06:20:08.698205 8171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:41627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:41167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.130
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:08.698366 8171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:08.698531 8171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:08.703199 8177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:08.703227 8179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:08.703368 8171 server_base.cc:1034] running on GCE node
W20260926 06:20:08.703421 8176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:08.703609 8171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:08.703799 8171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:08.704957 8171 hybrid_clock.cc:648] HybridClock initialized: now 1790403608704941 us; error 29 us; skew 500 ppm
Sep 26 06:20:08 dist-test-slave-pd0t krb5kdc[7930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403608, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:08.713086 8171 init.cc:377] Logged in from keytab as kudu/127.4.202.130@KRBTEST.COM (short username kudu)
I20260926 06:20:08.713526 8171 webserver.cc:460] Webserver started at http://127.4.202.130:45769/ using document root <none> and password file <none>
I20260926 06:20:08.713738 8171 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:08.713816 8171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:08.713963 8171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:08.715014 8171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/data/instance:
uuid: "78ebb74ea35745bda24665d95e5ad113"
format_stamp: "Formatted at 2026-09-26 06:20:08 on dist-test-slave-pd0t"
server_key: "37d577d313696ffa6e393fb5542ea3b9"
server_key_iv: "04bd12984512a4932cfd343517d706b3"
server_key_version: "encryptionkey@0"
I20260926 06:20:08.715353 8171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance:
uuid: "78ebb74ea35745bda24665d95e5ad113"
format_stamp: "Formatted at 2026-09-26 06:20:08 on dist-test-slave-pd0t"
server_key: "37d577d313696ffa6e393fb5542ea3b9"
server_key_iv: "04bd12984512a4932cfd343517d706b3"
server_key_version: "encryptionkey@0"
I20260926 06:20:08.716663 8171 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:20:08.717366 8186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:08.717537 8171 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:20:08.717634 8171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/wal
uuid: "78ebb74ea35745bda24665d95e5ad113"
format_stamp: "Formatted at 2026-09-26 06:20:08 on dist-test-slave-pd0t"
server_key: "37d577d313696ffa6e393fb5542ea3b9"
server_key_iv: "04bd12984512a4932cfd343517d706b3"
server_key_version: "encryptionkey@0"
I20260926 06:20:08.717710 8171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:08.725569 8171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:08.727685 8171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:08.727823 8171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:08.728019 8171 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:20:08.728274 8171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:20:08.728323 8171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:08.728358 8171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:20:08.728384 8171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:08.733917 8171 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.130:45861
I20260926 06:20:08.733951 8299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.130:45861 every 8 connection(s)
I20260926 06:20:08.734251 8171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
Sep 26 06:20:08 dist-test-slave-pd0t krb5kdc[7930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403608, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:20:08.742216 8033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:08.734894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:56589 (local address 127.4.202.190:41167)
0926 06:20:08.734931 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:08.734933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:08.735103 (+ 170us) server_negotiation.cc:407] Connection header received
0926 06:20:08.735534 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:08.735538 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:08.735546 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:08.735599 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:08.736428 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:08.736899 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:08.737860 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:08.738058 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:08.738107 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:08.740043 (+ 1936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:08.740045 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:08.740045 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:08.740051 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:08.741131 (+ 1080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:08.741468 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:08.741469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:08.741469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:08.741515 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:08.741706 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:08.741706 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:08.741706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:08.741790 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:08.741836 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:08.742065 (+ 229us) server_negotiation.cc:299] Negotiation successful
0926 06:20:08.742131 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 06:20:08.742308 8302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:08.734852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41167 (local address 127.4.202.130:56589)
0926 06:20:08.734995 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:08.735030 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:08.735458 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:08.735663 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:08.735666 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:08.736164 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:08.736345 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:08.736349 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:08.736975 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:08.736976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:08.737776 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:08.737778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:08.738126 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:08.738126 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:08.738218 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:08.738741 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:08.738760 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:08.739962 (+ 1202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:08.741198 (+ 1236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:08.741202 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:08.741204 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:08.741402 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:08.741568 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:08.741568 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:08.741569 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:08.741655 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:08.741840 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:08.741843 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:08.741949 (+ 106us) client_negotiation.cc:769] Sending connection context
0926 06:20:08.742039 (+ 90us) client_negotiation.cc:240] Negotiation successful
0926 06:20:08.742150 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":79,"threads_started":1}
I20260926 06:20:08.742363 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 8171
I20260926 06:20:08.742476 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance
I20260926 06:20:08.742709 4906 external_mini_cluster.cc:1428] Setting key 1dff5df9394345d04413159f7e048993
I20260926 06:20:08.742767 8300 heartbeater.cc:347] Connected to a master server at 127.4.202.190:41167
I20260926 06:20:08.742859 8300 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:08.743014 8300 heartbeater.cc:511] Master 127.4.202.190:41167 requested a full tablet report, sending...
I20260926 06:20:08.743369 7975 ts_manager.cc:194] Registered new tserver with Master: 78ebb74ea35745bda24665d95e5ad113 (127.4.202.130:45861)
I20260926 06:20:08.743757 7975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.130@KRBTEST.COM'} at 127.4.202.130:56589
WARNING: no policy specified for kudu/127.4.202.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
I20260926 06:20:08.766369 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.131:0
--local_ip_for_outbound_sockets=127.4.202.131
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:41167
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:41627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:08.861783 8305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:08.861958 8305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:08.861979 8305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:08.863143 8305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:08.863181 8305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:08.863209 8305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.131
I20260926 06:20:08.864553 8305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:41627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:41167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.131
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:08.864722 8305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:08.864876 8305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:08.869704 8311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:08.869694 8313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:08.869750 8310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:08.869819 8305 server_base.cc:1034] running on GCE node
I20260926 06:20:08.870070 8305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:08.870317 8305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:08.871455 8305 hybrid_clock.cc:648] HybridClock initialized: now 1790403608871447 us; error 26 us; skew 500 ppm
Sep 26 06:20:08 dist-test-slave-pd0t krb5kdc[7930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403608, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:08.873797 8305 init.cc:377] Logged in from keytab as kudu/127.4.202.131@KRBTEST.COM (short username kudu)
I20260926 06:20:08.874231 8305 webserver.cc:460] Webserver started at http://127.4.202.131:38495/ using document root <none> and password file <none>
I20260926 06:20:08.874390 8305 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:08.874430 8305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:08.874511 8305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:08.875556 8305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/data/instance:
uuid: "0159d94d5c344e8ca33c05b9164efb6f"
format_stamp: "Formatted at 2026-09-26 06:20:08 on dist-test-slave-pd0t"
server_key: "27a8d7de63f52fb9dcb89e7bbd4df1bb"
server_key_iv: "7b047c8eefb09899778cc4308d7e887f"
server_key_version: "encryptionkey@0"
I20260926 06:20:08.875828 8305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance:
uuid: "0159d94d5c344e8ca33c05b9164efb6f"
format_stamp: "Formatted at 2026-09-26 06:20:08 on dist-test-slave-pd0t"
server_key: "27a8d7de63f52fb9dcb89e7bbd4df1bb"
server_key_iv: "7b047c8eefb09899778cc4308d7e887f"
server_key_version: "encryptionkey@0"
I20260926 06:20:08.877022 8305 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 06:20:08.877619 8320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:08.877789 8305 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:20:08.877883 8305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/wal
uuid: "0159d94d5c344e8ca33c05b9164efb6f"
format_stamp: "Formatted at 2026-09-26 06:20:08 on dist-test-slave-pd0t"
server_key: "27a8d7de63f52fb9dcb89e7bbd4df1bb"
server_key_iv: "7b047c8eefb09899778cc4308d7e887f"
server_key_version: "encryptionkey@0"
I20260926 06:20:08.877964 8305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:08.887279 8305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:08.889158 8305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:08.889297 8305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:08.889494 8305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:20:08.889771 8305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:20:08.889830 8305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:08.889868 8305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:20:08.889916 8305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:08.895577 8305 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.131:46675
I20260926 06:20:08.895628 8433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.131:46675 every 8 connection(s)
I20260926 06:20:08.895918 8305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
I20260926 06:20:08.900692 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 8305
I20260926 06:20:08.900802 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance
I20260926 06:20:08.901005 4906 external_mini_cluster.cc:1428] Setting key 0d82fdf449df0593f692b4519767db91
Sep 26 06:20:08 dist-test-slave-pd0t krb5kdc[7930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403608, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:20:08.904687 8033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:08.896565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:53249 (local address 127.4.202.190:41167)
0926 06:20:08.896609 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:08.896611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:08.896779 (+ 168us) server_negotiation.cc:407] Connection header received
0926 06:20:08.897204 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:08.897205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:08.897227 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:08.897279 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:08.898119 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:08.898682 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:08.899614 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:08.899824 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:08.899929 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:08.902290 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:08.902291 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:08.902292 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:08.902297 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:08.903585 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:08.903934 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:08.903935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:08.903935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:08.903982 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:08.904176 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:08.904176 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:08.904176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:08.904264 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:08.904311 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:08.904541 (+ 230us) server_negotiation.cc:299] Negotiation successful
0926 06:20:08.904615 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:20:08.904879 8436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:08.896522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41167 (local address 127.4.202.131:53249)
0926 06:20:08.896659 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:08.896695 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:08.897125 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:08.897324 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:08.897345 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:08.897859 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:08.898036 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:08.898040 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:08.898753 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:08.898753 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:08.899498 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:08.899500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:08.899938 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:08.899939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:08.900061 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:08.900542 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:08.900550 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:08.902192 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:08.903654 (+ 1462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:08.903658 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:08.903660 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:08.903872 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:08.904036 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:08.904037 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:08.904037 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:08.904124 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:08.904314 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:08.904318 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:08.904438 (+ 120us) client_negotiation.cc:769] Sending connection context
0926 06:20:08.904514 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 06:20:08.904636 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":79,"threads_started":1}
I20260926 06:20:08.905339 8434 heartbeater.cc:347] Connected to a master server at 127.4.202.190:41167
I20260926 06:20:08.905411 8434 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:08.905570 8434 heartbeater.cc:511] Master 127.4.202.190:41167 requested a full tablet report, sending...
I20260926 06:20:08.905915 7975 ts_manager.cc:194] Registered new tserver with Master: 0159d94d5c344e8ca33c05b9164efb6f (127.4.202.131:46675)
I20260926 06:20:08.906244 7975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.131@KRBTEST.COM'} at 127.4.202.131:53249
I20260926 06:20:08.912782 4906 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 06:20:08 dist-test-slave-pd0t krb5kdc[7930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403608, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 26 06:20:08 dist-test-slave-pd0t krb5kdc[7930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403608, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:20:08.934091 8033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:08.927729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33454 (local address 127.4.202.190:41167)
0926 06:20:08.927788 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:08.927790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:08.927878 (+ 88us) server_negotiation.cc:407] Connection header received
0926 06:20:08.927928 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:08.927929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:08.927937 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:08.927984 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:08.928459 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:08.928915 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:08.929571 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:08.929737 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:08.929784 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:08.931920 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:08.931921 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:08.931921 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:08.931927 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:08.933081 (+ 1154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:08.933374 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:08.933375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:08.933375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:08.933431 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:08.933593 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:08.933594 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:08.933594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:08.933721 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:08.933791 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:08.933910 (+ 119us) server_negotiation.cc:299] Negotiation successful
0926 06:20:08.933967 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260926 06:20:08.934333 7975 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:33454
I20260926 06:20:08.934787 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 8037
I20260926 06:20:08.939193 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 8171
I20260926 06:20:08.943274 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 8305
I20260926 06:20:08.947326 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 7943
2026-09-26T06:20:08Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1783 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:20:08 dist-test-slave-pd0t krb5kdc[8446](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:20:08 dist-test-slave-pd0t krb5kdc[8446](info): set up 2 sockets
Sep 26 06:20:08 dist-test-slave-pd0t krb5kdc[8446](info): commencing operation
krb5kdc: starting...
W20260926 06:20:09.966542 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.000s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:20:10 dist-test-slave-pd0t krb5kdc[8446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:20:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:20:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.202.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
I20260926 06:20:10.075212 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:45903
--webserver_interface=127.4.202.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:44799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:45903
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:10.153206 8459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:10.153395 8459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:10.153416 8459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:10.154453 8459 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:20:10.154492 8459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:10.154506 8459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:10.154518 8459 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:20:10.154531 8459 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:20:10.156020 8459 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:44799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:45903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:45903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:10.156198 8459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:10.156348 8459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:10.161005 8467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:10.161142 8459 server_base.cc:1034] running on GCE node
W20260926 06:20:10.161103 8465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:10.161015 8464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:10.161448 8459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:10.161769 8459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:10.162923 8459 hybrid_clock.cc:648] HybridClock initialized: now 1790403610162907 us; error 43 us; skew 500 ppm
Sep 26 06:20:10 dist-test-slave-pd0t krb5kdc[8446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403610, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:10.165244 8459 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:20:10.165704 8459 webserver.cc:460] Webserver started at http://127.4.202.190:41597/ using document root <none> and password file <none>
I20260926 06:20:10.165921 8459 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:10.165992 8459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:10.166141 8459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:10.167191 8459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "e747811e116949719f94d2132464a67f"
format_stamp: "Formatted at 2026-09-26 06:20:10 on dist-test-slave-pd0t"
server_key: "964fb1dba199e6fff5a32bb68138d10a"
server_key_iv: "2ee83a76674997e68b60e27b45f3b9b9"
server_key_version: "encryptionkey@0"
I20260926 06:20:10.167544 8459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "e747811e116949719f94d2132464a67f"
format_stamp: "Formatted at 2026-09-26 06:20:10 on dist-test-slave-pd0t"
server_key: "964fb1dba199e6fff5a32bb68138d10a"
server_key_iv: "2ee83a76674997e68b60e27b45f3b9b9"
server_key_version: "encryptionkey@0"
I20260926 06:20:10.168828 8459 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:20:10.169513 8474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:10.169696 8459 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:20:10.169792 8459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "e747811e116949719f94d2132464a67f"
format_stamp: "Formatted at 2026-09-26 06:20:10 on dist-test-slave-pd0t"
server_key: "964fb1dba199e6fff5a32bb68138d10a"
server_key_iv: "2ee83a76674997e68b60e27b45f3b9b9"
server_key_version: "encryptionkey@0"
I20260926 06:20:10.169870 8459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:10.183482 8459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:10.185395 8459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:10.185551 8459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:10.189457 8459 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:45903
I20260926 06:20:10.189491 8526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:45903 every 8 connection(s)
I20260926 06:20:10.189949 8459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:20:10.190546 8527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:10.192631 8527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e747811e116949719f94d2132464a67f: Bootstrap starting.
I20260926 06:20:10.193188 8527 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e747811e116949719f94d2132464a67f: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:10.193475 8527 log.cc:824] T 00000000000000000000000000000000 P e747811e116949719f94d2132464a67f: Log is configured to *not* fsync() on all Append() calls
I20260926 06:20:10.194111 8527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e747811e116949719f94d2132464a67f: No bootstrap required, opened a new log
I20260926 06:20:10.195602 8527 raft_consensus.cc:348] T 00000000000000000000000000000000 P e747811e116949719f94d2132464a67f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e747811e116949719f94d2132464a67f" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45903 } }
I20260926 06:20:10.195729 8527 raft_consensus.cc:374] T 00000000000000000000000000000000 P e747811e116949719f94d2132464a67f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:10.195748 8527 raft_consensus.cc:729] T 00000000000000000000000000000000 P e747811e116949719f94d2132464a67f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e747811e116949719f94d2132464a67f, State: Initialized, Role: FOLLOWER
I20260926 06:20:10.195917 8527 consensus_queue.cc:260] T 00000000000000000000000000000000 P e747811e116949719f94d2132464a67f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e747811e116949719f94d2132464a67f" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45903 } }
I20260926 06:20:10.195976 8527 raft_consensus.cc:388] T 00000000000000000000000000000000 P e747811e116949719f94d2132464a67f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:20:10.195995 8527 raft_consensus.cc:482] T 00000000000000000000000000000000 P e747811e116949719f94d2132464a67f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:20:10.196063 8527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e747811e116949719f94d2132464a67f [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:10.196866 8527 raft_consensus.cc:504] T 00000000000000000000000000000000 P e747811e116949719f94d2132464a67f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e747811e116949719f94d2132464a67f" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45903 } }
I20260926 06:20:10.196997 8527 leader_election.cc:302] T 00000000000000000000000000000000 P e747811e116949719f94d2132464a67f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e747811e116949719f94d2132464a67f; no voters:
I20260926 06:20:10.197175 8527 leader_election.cc:288] T 00000000000000000000000000000000 P e747811e116949719f94d2132464a67f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:20:10.197223 8530 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e747811e116949719f94d2132464a67f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:20:10.197376 8530 raft_consensus.cc:686] T 00000000000000000000000000000000 P e747811e116949719f94d2132464a67f [term 1 LEADER]: Becoming Leader. State: Replica: e747811e116949719f94d2132464a67f, State: Running, Role: LEADER
I20260926 06:20:10.197422 8527 sys_catalog.cc:564] T 00000000000000000000000000000000 P e747811e116949719f94d2132464a67f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:20:10.197526 8530 consensus_queue.cc:237] T 00000000000000000000000000000000 P e747811e116949719f94d2132464a67f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e747811e116949719f94d2132464a67f" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45903 } }
I20260926 06:20:10.197873 8531 sys_catalog.cc:455] T 00000000000000000000000000000000 P e747811e116949719f94d2132464a67f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e747811e116949719f94d2132464a67f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e747811e116949719f94d2132464a67f" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45903 } } }
I20260926 06:20:10.197894 8532 sys_catalog.cc:455] T 00000000000000000000000000000000 P e747811e116949719f94d2132464a67f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e747811e116949719f94d2132464a67f. Latest consensus state: current_term: 1 leader_uuid: "e747811e116949719f94d2132464a67f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e747811e116949719f94d2132464a67f" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45903 } } }
I20260926 06:20:10.198061 8532 sys_catalog.cc:458] T 00000000000000000000000000000000 P e747811e116949719f94d2132464a67f [sys.catalog]: This master's current role is: LEADER
I20260926 06:20:10.198045 8531 sys_catalog.cc:458] T 00000000000000000000000000000000 P e747811e116949719f94d2132464a67f [sys.catalog]: This master's current role is: LEADER
I20260926 06:20:10.198506 8539 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:20:10.199039 8539 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:20:10.199330 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 8459
I20260926 06:20:10.199431 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:20:10.199599 4906 external_mini_cluster.cc:1428] Setting key bc659bf18bb3ccd5df89019cab12fb20
I20260926 06:20:10.200615 8539 catalog_manager.cc:1347] Generated new cluster ID: b8cb9663610147d392c466c60aaec6ef
I20260926 06:20:10.200664 8539 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:20:10 dist-test-slave-pd0t krb5kdc[8446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:20:10.209255 8539 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:20:10.209733 8539 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:20:10.211695 8549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:10.200140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45924 (local address 127.4.202.190:45903)
0926 06:20:10.200308 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:10.200317 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:10.200388 (+ 71us) server_negotiation.cc:407] Connection header received
0926 06:20:10.200525 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:10.200532 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:10.200581 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:10.200715 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:10.201572 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:10.203030 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:10.204109 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:10.204722 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:10.204820 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:10.207688 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:10.207692 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:10.207692 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:10.207703 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:10.209654 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:10.210063 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:10.210066 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:10.210066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:10.210178 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:10.210397 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:10.210398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:10.210398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:10.210677 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:10.210821 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:10.211245 (+ 424us) server_negotiation.cc:299] Negotiation successful
0926 06:20:10.211384 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":80,"threads_started":1}
I20260926 06:20:10.215971 8539 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e747811e116949719f94d2132464a67f: Generated new TSK 0
I20260926 06:20:10.216183 8539 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.202.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
I20260926 06:20:10.244918 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.129:0
--local_ip_for_outbound_sockets=127.4.202.129
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:45903
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:44799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:10.323629 8553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:10.323860 8553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:10.323895 8553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:10.324986 8553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:10.325045 8553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:10.325089 8553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.129
I20260926 06:20:10.326494 8553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:44799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:45903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.129
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:10.326742 8553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:10.326979 8553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:10.331671 8561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:10.331812 8553 server_base.cc:1034] running on GCE node
W20260926 06:20:10.331707 8559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:10.331676 8558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:10.332130 8553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:10.332341 8553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:10.333491 8553 hybrid_clock.cc:648] HybridClock initialized: now 1790403610333470 us; error 38 us; skew 500 ppm
Sep 26 06:20:10 dist-test-slave-pd0t krb5kdc[8446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403610, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:10.339370 8553 init.cc:377] Logged in from keytab as kudu/127.4.202.129@KRBTEST.COM (short username kudu)
I20260926 06:20:10.339805 8553 webserver.cc:460] Webserver started at http://127.4.202.129:46299/ using document root <none> and password file <none>
I20260926 06:20:10.340010 8553 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:10.340071 8553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:10.340209 8553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:10.341163 8553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-0/data/instance:
uuid: "177568621e6e4b72b90760a69f657c4d"
format_stamp: "Formatted at 2026-09-26 06:20:10 on dist-test-slave-pd0t"
server_key: "8884bd89212f958d2fcaff40cdc91c42"
server_key_iv: "56b4d858177601b4dc2ff99ffeac33fd"
server_key_version: "encryptionkey@0"
I20260926 06:20:10.341511 8553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance:
uuid: "177568621e6e4b72b90760a69f657c4d"
format_stamp: "Formatted at 2026-09-26 06:20:10 on dist-test-slave-pd0t"
server_key: "8884bd89212f958d2fcaff40cdc91c42"
server_key_iv: "56b4d858177601b4dc2ff99ffeac33fd"
server_key_version: "encryptionkey@0"
I20260926 06:20:10.342784 8553 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 06:20:10.343467 8568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:10.343645 8553 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:20:10.343736 8553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-0/wal
uuid: "177568621e6e4b72b90760a69f657c4d"
format_stamp: "Formatted at 2026-09-26 06:20:10 on dist-test-slave-pd0t"
server_key: "8884bd89212f958d2fcaff40cdc91c42"
server_key_iv: "56b4d858177601b4dc2ff99ffeac33fd"
server_key_version: "encryptionkey@0"
I20260926 06:20:10.343811 8553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:10.357887 8553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:10.359633 8553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:10.359771 8553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:10.359964 8553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:20:10.360226 8553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:20:10.360272 8553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:10.360308 8553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:20:10.360332 8553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:10.366026 8553 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.129:38855
I20260926 06:20:10.366060 8681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.129:38855 every 8 connection(s)
I20260926 06:20:10.366355 8553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
I20260926 06:20:10.369532 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 8553
I20260926 06:20:10.369637 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance
I20260926 06:20:10.369812 4906 external_mini_cluster.cc:1428] Setting key a2ae97a30b05bfa705e0d56ae7e33668
Sep 26 06:20:10 dist-test-slave-pd0t krb5kdc[8446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403610, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:20:10.376807 8684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:10.367004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:45903 (local address 127.4.202.129:42389)
0926 06:20:10.367142 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:10.367172 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:10.368074 (+ 902us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:10.368330 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:10.368333 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:10.368554 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:10.368714 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:10.368717 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:10.369405 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:10.369406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:10.370406 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:10.370409 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:10.371058 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:10.371059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:10.371150 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:10.371573 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:10.371581 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:10.373238 (+ 1657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:10.375479 (+ 2241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:10.375485 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:10.375486 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:10.375680 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:10.375906 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:10.375906 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:10.375906 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:10.375994 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:10.376371 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:10.376375 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:10.376484 (+ 109us) client_negotiation.cc:769] Sending connection context
0926 06:20:10.376559 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 06:20:10.376665 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":82,"threads_started":1}
I20260926 06:20:10.376839 8549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:10.367028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:42389 (local address 127.4.202.190:45903)
0926 06:20:10.367068 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:10.367070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:10.367256 (+ 186us) server_negotiation.cc:407] Connection header received
0926 06:20:10.368192 (+ 936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:10.368193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:10.368205 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:10.368265 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:10.368806 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:10.369297 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:10.370643 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:10.370979 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:10.371054 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:10.373398 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:10.373399 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:10.373400 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:10.373408 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:10.375378 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:10.375758 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:10.375759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:10.375760 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:10.375830 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:10.376064 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:10.376066 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:10.376066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:10.376259 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:10.376371 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:10.376629 (+ 258us) server_negotiation.cc:299] Negotiation successful
0926 06:20:10.376731 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 06:20:10.377362 8682 heartbeater.cc:347] Connected to a master server at 127.4.202.190:45903
I20260926 06:20:10.377473 8682 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:10.377677 8682 heartbeater.cc:511] Master 127.4.202.190:45903 requested a full tablet report, sending...
I20260926 06:20:10.378186 8491 ts_manager.cc:194] Registered new tserver with Master: 177568621e6e4b72b90760a69f657c4d (127.4.202.129:38855)
I20260926 06:20:10.378885 8491 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.129@KRBTEST.COM'} at 127.4.202.129:42389
WARNING: no policy specified for kudu/127.4.202.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
I20260926 06:20:10.395468 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.130:0
--local_ip_for_outbound_sockets=127.4.202.130
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:45903
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:44799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:10.478266 8687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:10.478473 8687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:10.478497 8687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:10.479739 8687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:10.479780 8687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:10.479806 8687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.130
I20260926 06:20:10.481204 8687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:44799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:45903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.130
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:10.481377 8687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:10.481537 8687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:10.486196 8693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:10.486204 8692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:10.486331 8687 server_base.cc:1034] running on GCE node
W20260926 06:20:10.486204 8695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:10.486681 8687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:10.486923 8687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:10.488061 8687 hybrid_clock.cc:648] HybridClock initialized: now 1790403610488051 us; error 26 us; skew 500 ppm
Sep 26 06:20:10 dist-test-slave-pd0t krb5kdc[8446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403610, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:10.490483 8687 init.cc:377] Logged in from keytab as kudu/127.4.202.130@KRBTEST.COM (short username kudu)
I20260926 06:20:10.490891 8687 webserver.cc:460] Webserver started at http://127.4.202.130:40385/ using document root <none> and password file <none>
I20260926 06:20:10.491065 8687 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:10.491104 8687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:10.491189 8687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:10.492116 8687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-1/data/instance:
uuid: "84482413aa274c3a8debfae2168f4980"
format_stamp: "Formatted at 2026-09-26 06:20:10 on dist-test-slave-pd0t"
server_key: "ac50aa2a6dbd88fe68367f6167542220"
server_key_iv: "4d6ec22046a87db36991dc32ce1ba9ad"
server_key_version: "encryptionkey@0"
I20260926 06:20:10.492412 8687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance:
uuid: "84482413aa274c3a8debfae2168f4980"
format_stamp: "Formatted at 2026-09-26 06:20:10 on dist-test-slave-pd0t"
server_key: "ac50aa2a6dbd88fe68367f6167542220"
server_key_iv: "4d6ec22046a87db36991dc32ce1ba9ad"
server_key_version: "encryptionkey@0"
I20260926 06:20:10.493590 8687 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 06:20:10.494217 8702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:10.494383 8687 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:10.494460 8687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-1/wal
uuid: "84482413aa274c3a8debfae2168f4980"
format_stamp: "Formatted at 2026-09-26 06:20:10 on dist-test-slave-pd0t"
server_key: "ac50aa2a6dbd88fe68367f6167542220"
server_key_iv: "4d6ec22046a87db36991dc32ce1ba9ad"
server_key_version: "encryptionkey@0"
I20260926 06:20:10.494510 8687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:10.501509 8687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:10.503239 8687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:10.503350 8687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:10.503506 8687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:20:10.503796 8687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:20:10.503829 8687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:10.503880 8687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:20:10.503912 8687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:10.509229 8687 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.130:38171
I20260926 06:20:10.509292 8815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.130:38171 every 8 connection(s)
I20260926 06:20:10.509591 8687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
I20260926 06:20:10.509859 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 8687
I20260926 06:20:10.509969 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance
I20260926 06:20:10.510172 4906 external_mini_cluster.cc:1428] Setting key 867a80004797a2d4421c554b4d7e080a
Sep 26 06:20:10 dist-test-slave-pd0t krb5kdc[8446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403610, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.202.131@KRBTEST.COM; defaulting to no policy
I20260926 06:20:10.519912 8549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:10.510341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:34681 (local address 127.4.202.190:45903)
0926 06:20:10.510388 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:10.510391 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:10.510739 (+ 348us) server_negotiation.cc:407] Connection header received
0926 06:20:10.512064 (+ 1325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:10.512065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:10.512075 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:10.512131 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:10.512989 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:10.513806 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:10.514844 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:10.515173 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:10.515255 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:10.517210 (+ 1955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:10.517212 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:10.517212 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:10.517220 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:10.518726 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:10.519115 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:10.519116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:10.519117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:10.519176 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:10.519377 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:10.519378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:10.519378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:10.519489 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:10.519535 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:10.519778 (+ 243us) server_negotiation.cc:299] Negotiation successful
0926 06:20:10.519850 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 06:20:10.520116 8818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:10.510345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:45903 (local address 127.4.202.130:34681)
0926 06:20:10.510559 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:10.510630 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:10.511935 (+ 1305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:10.512319 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:10.512323 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:10.512703 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:10.512898 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:10.512902 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:10.513895 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:10.513896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:10.514767 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:10.514769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:10.515271 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:10.515271 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:10.515361 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:10.515866 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:10.515873 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:10.517122 (+ 1249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:10.518798 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:10.518803 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:10.518804 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:10.519045 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:10.519246 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:10.519247 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:10.519247 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:10.519333 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:10.519540 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:10.519544 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:10.519642 (+ 98us) client_negotiation.cc:769] Sending connection context
0926 06:20:10.519717 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 06:20:10.519865 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":127,"threads_started":1}
I20260926 06:20:10.520632 8816 heartbeater.cc:347] Connected to a master server at 127.4.202.190:45903
I20260926 06:20:10.520725 8816 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:10.520879 8816 heartbeater.cc:511] Master 127.4.202.190:45903 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.131@KRBTEST.COM" created.
I20260926 06:20:10.521266 8491 ts_manager.cc:194] Registered new tserver with Master: 84482413aa274c3a8debfae2168f4980 (127.4.202.130:38171)
I20260926 06:20:10.521696 8491 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.130@KRBTEST.COM'} at 127.4.202.130:34681
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
I20260926 06:20:10.534890 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.131:0
--local_ip_for_outbound_sockets=127.4.202.131
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:45903
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:44799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:10.622694 8821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:10.622912 8821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:10.622952 8821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:10.624119 8821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:10.624176 8821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:10.624217 8821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.131
I20260926 06:20:10.625736 8821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:44799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:45903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.131
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:10.626024 8821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:10.626240 8821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:10.631018 8826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:10.631090 8827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:10.631184 8821 server_base.cc:1034] running on GCE node
W20260926 06:20:10.631018 8829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:10.631464 8821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:10.631690 8821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:10.632848 8821 hybrid_clock.cc:648] HybridClock initialized: now 1790403610632843 us; error 35 us; skew 500 ppm
Sep 26 06:20:10 dist-test-slave-pd0t krb5kdc[8446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403610, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:10.635193 8821 init.cc:377] Logged in from keytab as kudu/127.4.202.131@KRBTEST.COM (short username kudu)
I20260926 06:20:10.635529 8821 webserver.cc:460] Webserver started at http://127.4.202.131:44701/ using document root <none> and password file <none>
I20260926 06:20:10.635695 8821 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:10.635732 8821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:10.635813 8821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:10.636756 8821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-2/data/instance:
uuid: "c5ea6976604146b092ee911be6992339"
format_stamp: "Formatted at 2026-09-26 06:20:10 on dist-test-slave-pd0t"
server_key: "2b95dbff41db2c03e20dc268ac30fd87"
server_key_iv: "8fbe1ec40da91dca42efba8eac075ac7"
server_key_version: "encryptionkey@0"
I20260926 06:20:10.637040 8821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance:
uuid: "c5ea6976604146b092ee911be6992339"
format_stamp: "Formatted at 2026-09-26 06:20:10 on dist-test-slave-pd0t"
server_key: "2b95dbff41db2c03e20dc268ac30fd87"
server_key_iv: "8fbe1ec40da91dca42efba8eac075ac7"
server_key_version: "encryptionkey@0"
I20260926 06:20:10.638211 8821 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 06:20:10.638980 8836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:10.639160 8821 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:10.639252 8821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-2/wal
uuid: "c5ea6976604146b092ee911be6992339"
format_stamp: "Formatted at 2026-09-26 06:20:10 on dist-test-slave-pd0t"
server_key: "2b95dbff41db2c03e20dc268ac30fd87"
server_key_iv: "8fbe1ec40da91dca42efba8eac075ac7"
server_key_version: "encryptionkey@0"
I20260926 06:20:10.639323 8821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:10.646911 8821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:10.649093 8821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:10.649271 8821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:10.649477 8821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:20:10.649755 8821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:20:10.649789 8821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:10.649842 8821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:20:10.649878 8821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:10.655512 8821 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.131:44595
I20260926 06:20:10.655587 8949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.131:44595 every 8 connection(s)
I20260926 06:20:10.655864 8821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
I20260926 06:20:10.658809 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 8821
I20260926 06:20:10.658923 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance
I20260926 06:20:10.659096 4906 external_mini_cluster.cc:1428] Setting key 01bff1d56bf10629c827e842861ad7ad
Sep 26 06:20:10 dist-test-slave-pd0t krb5kdc[8446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403610, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:20:10.665127 8549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:10.656558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:34109 (local address 127.4.202.190:45903)
0926 06:20:10.656617 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:10.656619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:10.656828 (+ 209us) server_negotiation.cc:407] Connection header received
0926 06:20:10.657741 (+ 913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:10.657742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:10.657756 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:10.657825 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:10.658340 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:10.659233 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:10.660547 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:10.660777 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:10.660834 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:10.662835 (+ 2001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:10.662836 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:10.662837 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:10.662844 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:10.663977 (+ 1133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:10.664297 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:10.664298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:10.664298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:10.664344 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:10.664569 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:10.664569 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:10.664570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:10.664659 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:10.664714 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:10.664997 (+ 283us) server_negotiation.cc:299] Negotiation successful
0926 06:20:10.665066 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 06:20:10.665220 8952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:10.656523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:45903 (local address 127.4.202.131:34109)
0926 06:20:10.656662 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:10.656740 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:10.657634 (+ 894us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:10.657872 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:10.657875 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:10.658094 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:10.658257 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:10.658260 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:10.659345 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:10.659346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:10.660467 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:10.660469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:10.660849 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:10.660850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:10.660944 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:10.661342 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:10.661350 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:10.662749 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:10.664044 (+ 1295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:10.664049 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:10.664050 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:10.664238 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:10.664425 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:10.664426 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:10.664426 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:10.664517 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:10.664710 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:10.664714 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:10.664807 (+ 93us) client_negotiation.cc:769] Sending connection context
0926 06:20:10.664887 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 06:20:10.665042 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":72,"threads_started":1}
I20260926 06:20:10.665678 8950 heartbeater.cc:347] Connected to a master server at 127.4.202.190:45903
I20260926 06:20:10.665750 8950 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:10.665867 8950 heartbeater.cc:511] Master 127.4.202.190:45903 requested a full tablet report, sending...
I20260926 06:20:10.666162 8491 ts_manager.cc:194] Registered new tserver with Master: c5ea6976604146b092ee911be6992339 (127.4.202.131:44595)
I20260926 06:20:10.666540 8491 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.131@KRBTEST.COM'} at 127.4.202.131:34109
I20260926 06:20:10.670856 4906 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 06:20:10 dist-test-slave-pd0t krb5kdc[8446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 06:20:10 dist-test-slave-pd0t krb5kdc[8446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.129@KRBTEST.COM
I20260926 06:20:10.769933 8957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:10.761523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53078 (local address 127.4.202.129:38855)
0926 06:20:10.761780 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:10.761784 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:10.761794 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:20:10.761818 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:10.761820 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:10.761833 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:10.761893 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:10.762716 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:10.763447 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:10.764736 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:10.765058 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:10.765116 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:10.767347 (+ 2231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:10.767356 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:10.767356 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:10.767385 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:10.768979 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:10.769275 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:10.769277 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:10.769278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:10.769330 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:10.769507 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:10.769508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:10.769508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:10.769629 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:10.769729 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:10.769790 (+ 61us) server_negotiation.cc:299] Negotiation successful
0926 06:20:10.769848 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":101,"threads_started":1}
Sep 26 06:20:10 dist-test-slave-pd0t krb5kdc[8446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:20:10.777880 8549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:10.771197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45926 (local address 127.4.202.190:45903)
0926 06:20:10.771267 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:10.771270 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:10.771279 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:20:10.771375 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:10.771376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:10.771386 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:10.771459 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:10.771900 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:10.772430 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:10.773132 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:10.773334 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:10.773414 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:10.775624 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:10.775626 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:10.775626 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:10.775632 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:10.777004 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:10.777284 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:10.777284 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:10.777285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:10.777329 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:10.777473 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:10.777473 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:10.777474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:10.777590 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:10.777652 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:10.777739 (+ 87us) server_negotiation.cc:299] Negotiation successful
0926 06:20:10.777809 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260926 06:20:10.778126 8491 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45926
I20260926 06:20:10.778544 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 8553
I20260926 06:20:10.782828 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 8687
I20260926 06:20:10.787271 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 8821
I20260926 06:20:10.791240 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 8459
2026-09-26T06:20:10Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1844 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:20:12 dist-test-slave-pd0t krb5kdc[8962](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:20:12 dist-test-slave-pd0t krb5kdc[8962](info): set up 2 sockets
Sep 26 06:20:12 dist-test-slave-pd0t krb5kdc[8962](info): commencing operation
krb5kdc: starting...
W20260926 06:20:13.569828 4906 mini_kdc.cc:121] Time spent starting KDC: real 2.759s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260926 06:20:13.735685 4906 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.166s user 0.000s sys 0.002s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260926 06:20:13.945009 4906 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.209s user 0.000s sys 0.001s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260926 06:20:14.227319 4906 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.282s user 0.000s sys 0.003s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/krb5kdc/test-user.keytab.
W20260926 06:20:15.016937 4906 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.789s user 0.000s sys 0.002s
Sep 26 06:20:15 dist-test-slave-pd0t krb5kdc[8962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:20:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:20:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.202.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
I20260926 06:20:15.057498 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:33789
--webserver_interface=127.4.202.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:41423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:33789
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:15.139778 8975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:15.139957 8975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:15.139979 8975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:15.141019 8975 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:20:15.141052 8975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:15.141067 8975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:15.141078 8975 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:20:15.141090 8975 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:20:15.142547 8975 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:41423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:33789
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:33789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:15.142736 8975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:15.142892 8975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:15.147392 8981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:15.147487 8975 server_base.cc:1034] running on GCE node
W20260926 06:20:15.147550 8983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:15.147436 8980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:15.147898 8975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:15.148139 8975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:15.149282 8975 hybrid_clock.cc:648] HybridClock initialized: now 1790403615149272 us; error 29 us; skew 500 ppm
Sep 26 06:20:15 dist-test-slave-pd0t krb5kdc[8962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403615, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:15.151656 8975 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:20:15.152016 8975 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:15.152064 8975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:15.152247 8975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:15.153245 8975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "2540a8b227dc410292c1802cdbc7cfdb"
format_stamp: "Formatted at 2026-09-26 06:20:15 on dist-test-slave-pd0t"
server_key: "3a538546744b87d625cc0ff903f7b51b"
server_key_iv: "58d6618cfe452ea5a4ba3a2bacd52fbe"
server_key_version: "encryptionkey@0"
I20260926 06:20:15.153594 8975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "2540a8b227dc410292c1802cdbc7cfdb"
format_stamp: "Formatted at 2026-09-26 06:20:15 on dist-test-slave-pd0t"
server_key: "3a538546744b87d625cc0ff903f7b51b"
server_key_iv: "58d6618cfe452ea5a4ba3a2bacd52fbe"
server_key_version: "encryptionkey@0"
I20260926 06:20:15.154872 8975 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:20:15.155543 8989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:15.155721 8975 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:15.155792 8975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "2540a8b227dc410292c1802cdbc7cfdb"
format_stamp: "Formatted at 2026-09-26 06:20:15 on dist-test-slave-pd0t"
server_key: "3a538546744b87d625cc0ff903f7b51b"
server_key_iv: "58d6618cfe452ea5a4ba3a2bacd52fbe"
server_key_version: "encryptionkey@0"
I20260926 06:20:15.155880 8975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:15.170945 8975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:15.172724 8975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:15.172876 8975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:15.176806 8975 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:33789
I20260926 06:20:15.176860 9041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:33789 every 8 connection(s)
I20260926 06:20:15.177114 8975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:20:15.178004 9042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:15.180214 9042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2540a8b227dc410292c1802cdbc7cfdb: Bootstrap starting.
I20260926 06:20:15.180805 9042 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2540a8b227dc410292c1802cdbc7cfdb: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:15.181107 9042 log.cc:824] T 00000000000000000000000000000000 P 2540a8b227dc410292c1802cdbc7cfdb: Log is configured to *not* fsync() on all Append() calls
I20260926 06:20:15.181746 9042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2540a8b227dc410292c1802cdbc7cfdb: No bootstrap required, opened a new log
I20260926 06:20:15.181783 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 8975
I20260926 06:20:15.181878 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:20:15.182044 4906 external_mini_cluster.cc:1428] Setting key 1079af6c5e61adfc0fe625d329dd9f31
I20260926 06:20:15.184692 9042 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2540a8b227dc410292c1802cdbc7cfdb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2540a8b227dc410292c1802cdbc7cfdb" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 33789 } }
I20260926 06:20:15.184854 9042 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2540a8b227dc410292c1802cdbc7cfdb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:15.184885 9042 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2540a8b227dc410292c1802cdbc7cfdb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2540a8b227dc410292c1802cdbc7cfdb, State: Initialized, Role: FOLLOWER
I20260926 06:20:15.184991 9042 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2540a8b227dc410292c1802cdbc7cfdb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2540a8b227dc410292c1802cdbc7cfdb" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 33789 } }
I20260926 06:20:15.185058 9042 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2540a8b227dc410292c1802cdbc7cfdb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:20:15.185084 9042 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2540a8b227dc410292c1802cdbc7cfdb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:20:15.185128 9042 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2540a8b227dc410292c1802cdbc7cfdb [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:15.185775 9042 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2540a8b227dc410292c1802cdbc7cfdb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2540a8b227dc410292c1802cdbc7cfdb" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 33789 } }
I20260926 06:20:15.185897 9042 leader_election.cc:302] T 00000000000000000000000000000000 P 2540a8b227dc410292c1802cdbc7cfdb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2540a8b227dc410292c1802cdbc7cfdb; no voters:
I20260926 06:20:15.186082 9042 leader_election.cc:288] T 00000000000000000000000000000000 P 2540a8b227dc410292c1802cdbc7cfdb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:20:15.186113 9047 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2540a8b227dc410292c1802cdbc7cfdb [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:20:15.186213 9047 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2540a8b227dc410292c1802cdbc7cfdb [term 1 LEADER]: Becoming Leader. State: Replica: 2540a8b227dc410292c1802cdbc7cfdb, State: Running, Role: LEADER
I20260926 06:20:15.186350 9042 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2540a8b227dc410292c1802cdbc7cfdb [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:20:15.186352 9047 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2540a8b227dc410292c1802cdbc7cfdb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2540a8b227dc410292c1802cdbc7cfdb" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 33789 } }
I20260926 06:20:15.186880 9049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2540a8b227dc410292c1802cdbc7cfdb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2540a8b227dc410292c1802cdbc7cfdb. Latest consensus state: current_term: 1 leader_uuid: "2540a8b227dc410292c1802cdbc7cfdb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2540a8b227dc410292c1802cdbc7cfdb" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 33789 } } }
I20260926 06:20:15.186971 9049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2540a8b227dc410292c1802cdbc7cfdb [sys.catalog]: This master's current role is: LEADER
I20260926 06:20:15.187353 9048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2540a8b227dc410292c1802cdbc7cfdb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2540a8b227dc410292c1802cdbc7cfdb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2540a8b227dc410292c1802cdbc7cfdb" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 33789 } } }
I20260926 06:20:15.187433 9048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2540a8b227dc410292c1802cdbc7cfdb [sys.catalog]: This master's current role is: LEADER
I20260926 06:20:15.187585 9055 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:20:15.188076 9055 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:20:15.189604 9055 catalog_manager.cc:1347] Generated new cluster ID: 489977aa5b8b463baec988dd93ce20c8
I20260926 06:20:15.189651 9055 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:20:15 dist-test-slave-pd0t krb5kdc[8962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:20:15.193845 9046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:15.182632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57774 (local address 127.4.202.190:33789)
0926 06:20:15.182804 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:15.182813 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:15.182852 (+ 39us) server_negotiation.cc:407] Connection header received
0926 06:20:15.183326 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:15.183334 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:15.183392 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:15.183535 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:15.184124 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.185490 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.186443 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.187055 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.187133 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:15.190192 (+ 3059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:15.190196 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:15.190196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:15.190207 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:15.192069 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:15.192496 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:15.192499 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:15.192500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:15.192600 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:15.192827 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:15.192828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:15.192829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:15.193109 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:15.193242 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:15.193498 (+ 256us) server_negotiation.cc:299] Negotiation successful
0926 06:20:15.193628 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":69,"threads_started":1}
I20260926 06:20:15.205237 9055 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:20:15.205847 9055 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:20:15.217334 9055 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2540a8b227dc410292c1802cdbc7cfdb: Generated new TSK 0
I20260926 06:20:15.217500 9055 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.202.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
I20260926 06:20:15.241348 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.129:0
--local_ip_for_outbound_sockets=127.4.202.129
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:33789
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:41423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:15.320070 9068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:15.320261 9068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:15.320284 9068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:15.321456 9068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:15.321506 9068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:15.321533 9068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.129
I20260926 06:20:15.323098 9068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:41423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:33789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.129
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:15.323282 9068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:15.323559 9068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:15.328334 9073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:15.328421 9068 server_base.cc:1034] running on GCE node
W20260926 06:20:15.328323 9074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:15.328323 9076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:15.328697 9068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:15.328959 9068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:15.330102 9068 hybrid_clock.cc:648] HybridClock initialized: now 1790403615330088 us; error 29 us; skew 500 ppm
Sep 26 06:20:15 dist-test-slave-pd0t krb5kdc[8962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403615, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:15.332649 9068 init.cc:377] Logged in from keytab as kudu/127.4.202.129@KRBTEST.COM (short username kudu)
I20260926 06:20:15.332974 9068 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:15.333019 9068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:15.333184 9068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:15.334152 9068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-0/data/instance:
uuid: "4da400afe85b441096ee22a8d226ed68"
format_stamp: "Formatted at 2026-09-26 06:20:15 on dist-test-slave-pd0t"
server_key: "07ffd907862a970e2078f110f1cf4d02"
server_key_iv: "303b98d2165b60b44e4d44cb8462f7ee"
server_key_version: "encryptionkey@0"
I20260926 06:20:15.334491 9068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance:
uuid: "4da400afe85b441096ee22a8d226ed68"
format_stamp: "Formatted at 2026-09-26 06:20:15 on dist-test-slave-pd0t"
server_key: "07ffd907862a970e2078f110f1cf4d02"
server_key_iv: "303b98d2165b60b44e4d44cb8462f7ee"
server_key_version: "encryptionkey@0"
I20260926 06:20:15.335768 9068 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:20:15.336535 9082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:15.336796 9068 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:20:15.336937 9068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-0/wal
uuid: "4da400afe85b441096ee22a8d226ed68"
format_stamp: "Formatted at 2026-09-26 06:20:15 on dist-test-slave-pd0t"
server_key: "07ffd907862a970e2078f110f1cf4d02"
server_key_iv: "303b98d2165b60b44e4d44cb8462f7ee"
server_key_version: "encryptionkey@0"
I20260926 06:20:15.337036 9068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:15.374727 9068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:15.377032 9068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:15.377209 9068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:15.377468 9068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:20:15.377792 9068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:20:15.377853 9068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:15.377895 9068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:20:15.377925 9068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:15.383680 9068 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.129:37229
I20260926 06:20:15.383720 9195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.129:37229 every 8 connection(s)
I20260926 06:20:15.383980 9068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
I20260926 06:20:15.385838 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 9068
I20260926 06:20:15.385936 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance
I20260926 06:20:15.386121 4906 external_mini_cluster.cc:1428] Setting key 2dd5f32dac00bd240a52db3adbe56728
Sep 26 06:20:15 dist-test-slave-pd0t krb5kdc[8962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403615, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:20:15.394428 9046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:15.384661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:60407 (local address 127.4.202.190:33789)
0926 06:20:15.384728 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:15.384730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:15.384925 (+ 195us) server_negotiation.cc:407] Connection header received
0926 06:20:15.385418 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:15.385420 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:15.385431 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:15.385498 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:15.386725 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.387304 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.388745 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.388954 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.389013 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:15.391509 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:15.391510 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:15.391511 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:15.391517 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:15.392920 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:15.393438 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:15.393438 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:15.393439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:15.393490 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:15.393770 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:15.393770 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:15.393770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:15.393873 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:15.393926 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:15.394291 (+ 365us) server_negotiation.cc:299] Negotiation successful
0926 06:20:15.394364 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 06:20:15.394702 9198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:15.384658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:33789 (local address 127.4.202.129:60407)
0926 06:20:15.384799 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:15.384834 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:15.385282 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:15.385549 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:15.385552 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:15.385921 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:15.386534 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:15.386541 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.387404 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.387406 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:15.388657 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:15.388660 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.389032 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.389033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:15.389153 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:15.389678 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:15.389686 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:15.391413 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:15.393014 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:15.393021 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:15.393023 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:15.393359 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:15.393556 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:15.393557 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:15.393558 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:15.393693 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:15.393943 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:15.393949 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:15.394118 (+ 169us) client_negotiation.cc:769] Sending connection context
0926 06:20:15.394246 (+ 128us) client_negotiation.cc:240] Negotiation successful
0926 06:20:15.394438 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":79,"threads_started":1}
WARNING: no policy specified for kudu/127.4.202.130@KRBTEST.COM; defaulting to no policy
I20260926 06:20:15.395251 9196 heartbeater.cc:347] Connected to a master server at 127.4.202.190:33789
I20260926 06:20:15.395354 9196 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:15.395498 9196 heartbeater.cc:511] Master 127.4.202.190:33789 requested a full tablet report, sending...
I20260926 06:20:15.396082 9006 ts_manager.cc:194] Registered new tserver with Master: 4da400afe85b441096ee22a8d226ed68 (127.4.202.129:37229)
I20260926 06:20:15.396776 9006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.129@KRBTEST.COM'} at 127.4.202.129:60407
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
I20260926 06:20:15.411433 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.130:0
--local_ip_for_outbound_sockets=127.4.202.130
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:33789
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:41423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:15.491629 9201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:15.491858 9201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:15.491899 9201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:15.493108 9201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:15.493176 9201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:15.493222 9201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.130
I20260926 06:20:15.494796 9201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:41423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:33789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.130
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:15.495039 9201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:15.495249 9201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:15.500013 9209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:15.500025 9207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:15.500016 9206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:15.500231 9201 server_base.cc:1034] running on GCE node
I20260926 06:20:15.500454 9201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:15.500694 9201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:15.501859 9201 hybrid_clock.cc:648] HybridClock initialized: now 1790403615501852 us; error 35 us; skew 500 ppm
Sep 26 06:20:15 dist-test-slave-pd0t krb5kdc[8962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403615, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:15.504283 9201 init.cc:377] Logged in from keytab as kudu/127.4.202.130@KRBTEST.COM (short username kudu)
I20260926 06:20:15.504607 9201 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:15.504676 9201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:15.504822 9201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:15.505829 9201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-1/data/instance:
uuid: "acfa18225c5b4a5ca3ea16defd40c0b6"
format_stamp: "Formatted at 2026-09-26 06:20:15 on dist-test-slave-pd0t"
server_key: "26f2a18c42cbd13cf65095871f4607f2"
server_key_iv: "252073b650a444b09aedbf445f1a54ea"
server_key_version: "encryptionkey@0"
I20260926 06:20:15.506179 9201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance:
uuid: "acfa18225c5b4a5ca3ea16defd40c0b6"
format_stamp: "Formatted at 2026-09-26 06:20:15 on dist-test-slave-pd0t"
server_key: "26f2a18c42cbd13cf65095871f4607f2"
server_key_iv: "252073b650a444b09aedbf445f1a54ea"
server_key_version: "encryptionkey@0"
I20260926 06:20:15.507489 9201 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:20:15.508277 9215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:15.508481 9201 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:20:15.508581 9201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-1/wal
uuid: "acfa18225c5b4a5ca3ea16defd40c0b6"
format_stamp: "Formatted at 2026-09-26 06:20:15 on dist-test-slave-pd0t"
server_key: "26f2a18c42cbd13cf65095871f4607f2"
server_key_iv: "252073b650a444b09aedbf445f1a54ea"
server_key_version: "encryptionkey@0"
I20260926 06:20:15.508658 9201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:15.527772 9201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:15.530658 9201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:15.530781 9201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:15.530941 9201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:20:15.531229 9201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:20:15.531262 9201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:15.531319 9201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:20:15.531347 9201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:15.537005 9201 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.130:35917
I20260926 06:20:15.537057 9328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.130:35917 every 8 connection(s)
I20260926 06:20:15.537333 9201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
Sep 26 06:20:15 dist-test-slave-pd0t krb5kdc[8962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403615, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:20:15.545648 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 9201
I20260926 06:20:15.545753 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance
I20260926 06:20:15.545990 4906 external_mini_cluster.cc:1428] Setting key 0cd88ba668e1fb16dc7abfad356c2dd8
I20260926 06:20:15.546422 9046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:15.538062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:39129 (local address 127.4.202.190:33789)
0926 06:20:15.538127 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:15.538129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:15.538314 (+ 185us) server_negotiation.cc:407] Connection header received
0926 06:20:15.538868 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:15.538869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:15.538881 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:15.538944 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:15.539886 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.540380 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.541432 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.541627 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.541710 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:15.543797 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:15.543798 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:15.543799 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:15.543805 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:15.545019 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:15.545368 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:15.545369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:15.545370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:15.545416 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:15.545721 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:15.545722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:15.545722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:15.545871 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:15.545926 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:15.546247 (+ 321us) server_negotiation.cc:299] Negotiation successful
0926 06:20:15.546330 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 06:20:15.546619 9331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:15.538044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:33789 (local address 127.4.202.130:39129)
0926 06:20:15.538194 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:15.538227 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:15.538767 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:15.539013 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:15.539016 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:15.539261 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:15.539775 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:15.539780 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.540471 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.540472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:15.541335 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:15.541337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.541719 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.541720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:15.541837 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:15.542306 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:15.542314 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:15.543713 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:15.545087 (+ 1374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:15.545093 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:15.545094 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:15.545304 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:15.545491 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:15.545492 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:15.545493 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:15.545633 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:15.545926 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:15.545932 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:15.546090 (+ 158us) client_negotiation.cc:769] Sending connection context
0926 06:20:15.546201 (+ 111us) client_negotiation.cc:240] Negotiation successful
0926 06:20:15.546380 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":80,"threads_started":1}
I20260926 06:20:15.547144 9329 heartbeater.cc:347] Connected to a master server at 127.4.202.190:33789
I20260926 06:20:15.547273 9329 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:15.547425 9329 heartbeater.cc:511] Master 127.4.202.190:33789 requested a full tablet report, sending...
I20260926 06:20:15.547780 9006 ts_manager.cc:194] Registered new tserver with Master: acfa18225c5b4a5ca3ea16defd40c0b6 (127.4.202.130:35917)
I20260926 06:20:15.548142 9006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.130@KRBTEST.COM'} at 127.4.202.130:39129
WARNING: no policy specified for kudu/127.4.202.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
I20260926 06:20:15.569687 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.131:0
--local_ip_for_outbound_sockets=127.4.202.131
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:33789
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:41423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:15.647867 9334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:15.648077 9334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:15.648116 9334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:15.649237 9334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:15.649305 9334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:15.649350 9334 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.131
I20260926 06:20:15.650836 9334 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:41423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:33789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.131
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:15.651090 9334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:15.651296 9334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:15.656090 9342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:15.656100 9339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:15.656095 9340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:15.656368 9334 server_base.cc:1034] running on GCE node
I20260926 06:20:15.656586 9334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:15.656816 9334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:15.657982 9334 hybrid_clock.cc:648] HybridClock initialized: now 1790403615657978 us; error 42 us; skew 500 ppm
Sep 26 06:20:15 dist-test-slave-pd0t krb5kdc[8962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403615, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:15.660365 9334 init.cc:377] Logged in from keytab as kudu/127.4.202.131@KRBTEST.COM (short username kudu)
I20260926 06:20:15.660679 9334 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:15.660761 9334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:15.660902 9334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:15.661844 9334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-2/data/instance:
uuid: "65e675b6f229423f836afdbfed785fd4"
format_stamp: "Formatted at 2026-09-26 06:20:15 on dist-test-slave-pd0t"
server_key: "0129288c79f033e8f00c74f36213288b"
server_key_iv: "b3c105277580bf5b2220da7058272ac2"
server_key_version: "encryptionkey@0"
I20260926 06:20:15.662190 9334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance:
uuid: "65e675b6f229423f836afdbfed785fd4"
format_stamp: "Formatted at 2026-09-26 06:20:15 on dist-test-slave-pd0t"
server_key: "0129288c79f033e8f00c74f36213288b"
server_key_iv: "b3c105277580bf5b2220da7058272ac2"
server_key_version: "encryptionkey@0"
I20260926 06:20:15.663518 9334 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:20:15.664207 9348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:15.664402 9334 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:20:15.664496 9334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-2/wal
uuid: "65e675b6f229423f836afdbfed785fd4"
format_stamp: "Formatted at 2026-09-26 06:20:15 on dist-test-slave-pd0t"
server_key: "0129288c79f033e8f00c74f36213288b"
server_key_iv: "b3c105277580bf5b2220da7058272ac2"
server_key_version: "encryptionkey@0"
I20260926 06:20:15.664573 9334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:15.677488 9334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:15.679286 9334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:15.679436 9334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:15.679637 9334 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:20:15.679919 9334 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:20:15.679975 9334 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:15.680014 9334 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:20:15.680042 9334 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:15.686146 9334 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.131:34281
I20260926 06:20:15.686216 9461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.131:34281 every 8 connection(s)
I20260926 06:20:15.686498 9334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
Sep 26 06:20:15 dist-test-slave-pd0t krb5kdc[8962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403615, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:20:15.693699 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 9334
I20260926 06:20:15.693827 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance
I20260926 06:20:15.694007 4906 external_mini_cluster.cc:1428] Setting key 2b0302a653da19c2da265ed9483902a1
I20260926 06:20:15.695369 9046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:15.687153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:38297 (local address 127.4.202.190:33789)
0926 06:20:15.687220 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:15.687221 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:15.687448 (+ 227us) server_negotiation.cc:407] Connection header received
0926 06:20:15.687916 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:15.687917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:15.687928 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:15.687979 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:15.688823 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.689320 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.690283 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.690488 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.690572 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:15.692541 (+ 1969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:15.692542 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:15.692543 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:15.692550 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:15.693753 (+ 1203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:15.694308 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:15.694310 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:15.694310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:15.694371 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:15.694681 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:15.694682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:15.694682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:15.694831 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:15.694908 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:15.695189 (+ 281us) server_negotiation.cc:299] Negotiation successful
0926 06:20:15.695283 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 06:20:15.695526 9464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:15.687138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:33789 (local address 127.4.202.131:38297)
0926 06:20:15.687315 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:15.687353 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:15.687811 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:15.688042 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:15.688046 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:15.688298 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:15.688724 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:15.688728 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.689410 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.689411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:15.690170 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:15.690172 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.690599 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.690601 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:15.690719 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:15.691172 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:15.691180 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:15.692445 (+ 1265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:15.693842 (+ 1397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:15.693849 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:15.693851 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:15.694112 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:15.694435 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:15.694436 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:15.694437 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:15.694567 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:15.694907 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:15.694914 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:15.695059 (+ 145us) client_negotiation.cc:769] Sending connection context
0926 06:20:15.695167 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 06:20:15.695309 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":87,"threads_started":1}
I20260926 06:20:15.695979 9462 heartbeater.cc:347] Connected to a master server at 127.4.202.190:33789
I20260926 06:20:15.696063 9462 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:15.696164 9462 heartbeater.cc:511] Master 127.4.202.190:33789 requested a full tablet report, sending...
I20260926 06:20:15.696545 9006 ts_manager.cc:194] Registered new tserver with Master: 65e675b6f229423f836afdbfed785fd4 (127.4.202.131:34281)
I20260926 06:20:15.696918 9006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.131@KRBTEST.COM'} at 127.4.202.131:38297
I20260926 06:20:15.705670 4906 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:20:15.713052 9046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:15.706958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57782 (local address 127.4.202.190:33789)
0926 06:20:15.707050 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:15.707052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:15.707155 (+ 103us) server_negotiation.cc:407] Connection header received
0926 06:20:15.707268 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:15.707269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:15.707277 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:15.707340 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:15.707900 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.708355 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.709098 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.709286 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.709392 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:15.710520 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:15.710522 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:15.710522 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:15.710528 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:15.711807 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:15.712168 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:15.712169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:15.712170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:15.712271 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:15.712503 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:15.712504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:15.712504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:15.712650 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:15.712754 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:15.712885 (+ 131us) server_negotiation.cc:299] Negotiation successful
0926 06:20:15.712958 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 06:20:15.714406 9006 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57782:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:20:15.715981 9006 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:20:15.722167 9474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:15.718411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58682 (local address 127.4.202.129:37229)
0926 06:20:15.718625 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:15.718629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:15.718639 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:20:15.718681 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:15.718683 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:15.718697 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:15.718773 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:20:15.718931 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.720010 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.721210 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.721771 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.721827 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:15.721904 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:15.722031 (+ 127us) server_negotiation.cc:299] Negotiation successful
0926 06:20:15.722079 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":68,"threads_started":1}
I20260926 06:20:15.722183 9471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:15.718184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:37229 (local address 127.0.0.1:58682)
0926 06:20:15.718537 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:15.718548 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:15.718623 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:15.718779 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:15.718780 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:15.718781 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:20:15.718846 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:15.718847 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.720108 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.720109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:15.721144 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:15.721145 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.721857 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.721858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:15.721935 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:15.721948 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 06:20:15.722013 (+ 65us) client_negotiation.cc:240] Negotiation successful
0926 06:20:15.722078 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":9,"thread_start_us":50,"threads_started":1}
I20260926 06:20:15.722436 9475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:15.718411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53838 (local address 127.4.202.130:35917)
0926 06:20:15.718928 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:15.718932 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:15.718940 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:20:15.718976 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:15.718977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:15.718993 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:15.719068 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:20:15.719312 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.720279 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.721289 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.721777 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.721877 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:15.721932 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:15.722311 (+ 379us) server_negotiation.cc:299] Negotiation successful
0926 06:20:15.722358 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":79,"threads_started":1}
I20260926 06:20:15.722177 9472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:15.718185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:35917 (local address 127.0.0.1:53838)
0926 06:20:15.718547 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:15.718556 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:15.718623 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:15.719148 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:15.719149 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:15.719151 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:20:15.719194 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:15.719196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.720370 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.720371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:15.721217 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:15.721219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.721887 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.721889 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:15.721948 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:15.721952 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:20:15.722019 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 06:20:15.722089 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":24,"thread_start_us":55,"threads_started":1}
I20260926 06:20:15.722913 9130 tablet_service.cc:1467] Processing CreateTablet for tablet bccb10ed592c4b5582e1e5a6a5492782 (DEFAULT_TABLE table=test-table [id=c22fcdea047f492086da7a5738f404e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:20:15.723088 9470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:15.718152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:34281 (local address 127.0.0.1:37604)
0926 06:20:15.718241 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:15.718277 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:15.718336 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:15.718770 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:15.718773 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:15.718775 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:20:15.718866 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:15.718867 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.719875 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.719876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:15.721137 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:15.721139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.722874 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.722876 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:15.722948 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:15.722952 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:20:15.722985 (+ 33us) client_negotiation.cc:240] Negotiation successful
0926 06:20:15.723024 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"mutex_wait_us":6,"thread_start_us":48,"threads_started":1}
I20260926 06:20:15.723263 9130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bccb10ed592c4b5582e1e5a6a5492782. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:15.723345 9263 tablet_service.cc:1467] Processing CreateTablet for tablet bccb10ed592c4b5582e1e5a6a5492782 (DEFAULT_TABLE table=test-table [id=c22fcdea047f492086da7a5738f404e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:20:15.723511 9473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:15.718382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37604 (local address 127.4.202.131:34281)
0926 06:20:15.718547 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:15.718552 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:15.718563 (+ 11us) server_negotiation.cc:407] Connection header received
0926 06:20:15.718628 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:15.718630 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:15.718647 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:15.718742 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:20:15.718947 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.719755 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.721936 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.722723 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.723300 (+ 577us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:15.723363 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:15.723388 (+ 25us) server_negotiation.cc:299] Negotiation successful
0926 06:20:15.723433 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":83,"threads_started":1}
I20260926 06:20:15.723618 9263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bccb10ed592c4b5582e1e5a6a5492782. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:15.726660 9476 tablet_bootstrap.cc:492] T bccb10ed592c4b5582e1e5a6a5492782 P 4da400afe85b441096ee22a8d226ed68: Bootstrap starting.
I20260926 06:20:15.726660 9477 tablet_bootstrap.cc:492] T bccb10ed592c4b5582e1e5a6a5492782 P acfa18225c5b4a5ca3ea16defd40c0b6: Bootstrap starting.
I20260926 06:20:15.727061 9396 tablet_service.cc:1467] Processing CreateTablet for tablet bccb10ed592c4b5582e1e5a6a5492782 (DEFAULT_TABLE table=test-table [id=c22fcdea047f492086da7a5738f404e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:20:15.727267 9396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bccb10ed592c4b5582e1e5a6a5492782. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:15.727298 9477 tablet_bootstrap.cc:654] T bccb10ed592c4b5582e1e5a6a5492782 P acfa18225c5b4a5ca3ea16defd40c0b6: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:15.727325 9476 tablet_bootstrap.cc:654] T bccb10ed592c4b5582e1e5a6a5492782 P 4da400afe85b441096ee22a8d226ed68: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:15.727959 9476 log.cc:824] T bccb10ed592c4b5582e1e5a6a5492782 P 4da400afe85b441096ee22a8d226ed68: Log is configured to *not* fsync() on all Append() calls
I20260926 06:20:15.727973 9477 log.cc:824] T bccb10ed592c4b5582e1e5a6a5492782 P acfa18225c5b4a5ca3ea16defd40c0b6: Log is configured to *not* fsync() on all Append() calls
I20260926 06:20:15.728713 9476 tablet_bootstrap.cc:492] T bccb10ed592c4b5582e1e5a6a5492782 P 4da400afe85b441096ee22a8d226ed68: No bootstrap required, opened a new log
I20260926 06:20:15.728776 9476 ts_tablet_manager.cc:1397] T bccb10ed592c4b5582e1e5a6a5492782 P 4da400afe85b441096ee22a8d226ed68: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:20:15.729002 9477 tablet_bootstrap.cc:492] T bccb10ed592c4b5582e1e5a6a5492782 P acfa18225c5b4a5ca3ea16defd40c0b6: No bootstrap required, opened a new log
I20260926 06:20:15.729054 9477 ts_tablet_manager.cc:1397] T bccb10ed592c4b5582e1e5a6a5492782 P acfa18225c5b4a5ca3ea16defd40c0b6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:20:15.729574 9480 tablet_bootstrap.cc:492] T bccb10ed592c4b5582e1e5a6a5492782 P 65e675b6f229423f836afdbfed785fd4: Bootstrap starting.
I20260926 06:20:15.730188 9480 tablet_bootstrap.cc:654] T bccb10ed592c4b5582e1e5a6a5492782 P 65e675b6f229423f836afdbfed785fd4: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:15.730822 9480 log.cc:824] T bccb10ed592c4b5582e1e5a6a5492782 P 65e675b6f229423f836afdbfed785fd4: Log is configured to *not* fsync() on all Append() calls
I20260926 06:20:15.731063 9477 raft_consensus.cc:348] T bccb10ed592c4b5582e1e5a6a5492782 P acfa18225c5b4a5ca3ea16defd40c0b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65e675b6f229423f836afdbfed785fd4" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 34281 } } peers { permanent_uuid: "4da400afe85b441096ee22a8d226ed68" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37229 } } peers { permanent_uuid: "acfa18225c5b4a5ca3ea16defd40c0b6" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 35917 } }
I20260926 06:20:15.731133 9476 raft_consensus.cc:348] T bccb10ed592c4b5582e1e5a6a5492782 P 4da400afe85b441096ee22a8d226ed68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65e675b6f229423f836afdbfed785fd4" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 34281 } } peers { permanent_uuid: "4da400afe85b441096ee22a8d226ed68" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37229 } } peers { permanent_uuid: "acfa18225c5b4a5ca3ea16defd40c0b6" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 35917 } }
I20260926 06:20:15.731227 9477 raft_consensus.cc:374] T bccb10ed592c4b5582e1e5a6a5492782 P acfa18225c5b4a5ca3ea16defd40c0b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:15.731262 9476 raft_consensus.cc:374] T bccb10ed592c4b5582e1e5a6a5492782 P 4da400afe85b441096ee22a8d226ed68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:15.731264 9477 raft_consensus.cc:729] T bccb10ed592c4b5582e1e5a6a5492782 P acfa18225c5b4a5ca3ea16defd40c0b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfa18225c5b4a5ca3ea16defd40c0b6, State: Initialized, Role: FOLLOWER
I20260926 06:20:15.731331 9476 raft_consensus.cc:729] T bccb10ed592c4b5582e1e5a6a5492782 P 4da400afe85b441096ee22a8d226ed68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4da400afe85b441096ee22a8d226ed68, State: Initialized, Role: FOLLOWER
I20260926 06:20:15.731397 9477 consensus_queue.cc:260] T bccb10ed592c4b5582e1e5a6a5492782 P acfa18225c5b4a5ca3ea16defd40c0b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65e675b6f229423f836afdbfed785fd4" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 34281 } } peers { permanent_uuid: "4da400afe85b441096ee22a8d226ed68" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37229 } } peers { permanent_uuid: "acfa18225c5b4a5ca3ea16defd40c0b6" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 35917 } }
I20260926 06:20:15.731524 9476 consensus_queue.cc:260] T bccb10ed592c4b5582e1e5a6a5492782 P 4da400afe85b441096ee22a8d226ed68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65e675b6f229423f836afdbfed785fd4" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 34281 } } peers { permanent_uuid: "4da400afe85b441096ee22a8d226ed68" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37229 } } peers { permanent_uuid: "acfa18225c5b4a5ca3ea16defd40c0b6" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 35917 } }
I20260926 06:20:15.731598 9480 tablet_bootstrap.cc:492] T bccb10ed592c4b5582e1e5a6a5492782 P 65e675b6f229423f836afdbfed785fd4: No bootstrap required, opened a new log
I20260926 06:20:15.731654 9480 ts_tablet_manager.cc:1397] T bccb10ed592c4b5582e1e5a6a5492782 P 65e675b6f229423f836afdbfed785fd4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:20:15.731666 9477 ts_tablet_manager.cc:1428] T bccb10ed592c4b5582e1e5a6a5492782 P acfa18225c5b4a5ca3ea16defd40c0b6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 06:20:15.731704 9482 raft_consensus.cc:482] T bccb10ed592c4b5582e1e5a6a5492782 P 4da400afe85b441096ee22a8d226ed68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:20:15.731773 9329 heartbeater.cc:503] Master 127.4.202.190:33789 was elected leader, sending a full tablet report...
I20260926 06:20:15.731818 9482 raft_consensus.cc:504] T bccb10ed592c4b5582e1e5a6a5492782 P 4da400afe85b441096ee22a8d226ed68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65e675b6f229423f836afdbfed785fd4" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 34281 } } peers { permanent_uuid: "4da400afe85b441096ee22a8d226ed68" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37229 } } peers { permanent_uuid: "acfa18225c5b4a5ca3ea16defd40c0b6" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 35917 } }
I20260926 06:20:15.732477 9482 leader_election.cc:288] T bccb10ed592c4b5582e1e5a6a5492782 P 4da400afe85b441096ee22a8d226ed68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 65e675b6f229423f836afdbfed785fd4 (127.4.202.131:34281), acfa18225c5b4a5ca3ea16defd40c0b6 (127.4.202.130:35917)
I20260926 06:20:15.732647 9196 heartbeater.cc:503] Master 127.4.202.190:33789 was elected leader, sending a full tablet report...
I20260926 06:20:15.732833 9476 ts_tablet_manager.cc:1428] T bccb10ed592c4b5582e1e5a6a5492782 P 4da400afe85b441096ee22a8d226ed68: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260926 06:20:15.734020 9480 raft_consensus.cc:348] T bccb10ed592c4b5582e1e5a6a5492782 P 65e675b6f229423f836afdbfed785fd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65e675b6f229423f836afdbfed785fd4" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 34281 } } peers { permanent_uuid: "4da400afe85b441096ee22a8d226ed68" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37229 } } peers { permanent_uuid: "acfa18225c5b4a5ca3ea16defd40c0b6" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 35917 } }
I20260926 06:20:15.734169 9480 raft_consensus.cc:374] T bccb10ed592c4b5582e1e5a6a5492782 P 65e675b6f229423f836afdbfed785fd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:15.734210 9480 raft_consensus.cc:729] T bccb10ed592c4b5582e1e5a6a5492782 P 65e675b6f229423f836afdbfed785fd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65e675b6f229423f836afdbfed785fd4, State: Initialized, Role: FOLLOWER
I20260926 06:20:15.734342 9480 consensus_queue.cc:260] T bccb10ed592c4b5582e1e5a6a5492782 P 65e675b6f229423f836afdbfed785fd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65e675b6f229423f836afdbfed785fd4" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 34281 } } peers { permanent_uuid: "4da400afe85b441096ee22a8d226ed68" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37229 } } peers { permanent_uuid: "acfa18225c5b4a5ca3ea16defd40c0b6" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 35917 } }
I20260926 06:20:15.734634 9462 heartbeater.cc:503] Master 127.4.202.190:33789 was elected leader, sending a full tablet report...
I20260926 06:20:15.734627 9480 ts_tablet_manager.cc:1428] T bccb10ed592c4b5582e1e5a6a5492782 P 65e675b6f229423f836afdbfed785fd4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 06:20:15.735438 9198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:15.732312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:34281 (local address 127.4.202.129:36251)
0926 06:20:15.732350 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:15.732362 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:15.732407 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:15.732626 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:15.732627 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:15.732629 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:20:15.732689 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:15.732690 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.733521 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.733522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:15.734484 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:15.734486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.735224 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.735224 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:15.735302 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:15.735305 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:20:15.735366 (+ 61us) client_negotiation.cc:240] Negotiation successful
0926 06:20:15.735393 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260926 06:20:15.735469 9473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:15.732422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:36251 (local address 127.4.202.131:34281)
0926 06:20:15.732499 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:15.732501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:15.732508 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:20:15.732529 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:15.732530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:15.732539 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:15.732587 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:20:15.732776 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.733438 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.734572 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.735131 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.735188 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:15.735197 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:15.735374 (+ 177us) server_negotiation.cc:299] Negotiation successful
0926 06:20:15.735415 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 06:20:15.735710 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bccb10ed592c4b5582e1e5a6a5492782" candidate_uuid: "4da400afe85b441096ee22a8d226ed68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65e675b6f229423f836afdbfed785fd4" is_pre_election: true
I20260926 06:20:15.735822 9416 raft_consensus.cc:2454] T bccb10ed592c4b5582e1e5a6a5492782 P 65e675b6f229423f836afdbfed785fd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4da400afe85b441096ee22a8d226ed68 in term 0.
I20260926 06:20:15.735999 9084 leader_election.cc:302] T bccb10ed592c4b5582e1e5a6a5492782 P 4da400afe85b441096ee22a8d226ed68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4da400afe85b441096ee22a8d226ed68, 65e675b6f229423f836afdbfed785fd4; no voters:
I20260926 06:20:15.736119 9482 raft_consensus.cc:2781] T bccb10ed592c4b5582e1e5a6a5492782 P 4da400afe85b441096ee22a8d226ed68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:20:15.736152 9482 raft_consensus.cc:482] T bccb10ed592c4b5582e1e5a6a5492782 P 4da400afe85b441096ee22a8d226ed68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:20:15.736218 9482 raft_consensus.cc:3037] T bccb10ed592c4b5582e1e5a6a5492782 P 4da400afe85b441096ee22a8d226ed68 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:15.736930 9482 raft_consensus.cc:504] T bccb10ed592c4b5582e1e5a6a5492782 P 4da400afe85b441096ee22a8d226ed68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65e675b6f229423f836afdbfed785fd4" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 34281 } } peers { permanent_uuid: "4da400afe85b441096ee22a8d226ed68" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37229 } } peers { permanent_uuid: "acfa18225c5b4a5ca3ea16defd40c0b6" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 35917 } }
I20260926 06:20:15.737108 9482 leader_election.cc:288] T bccb10ed592c4b5582e1e5a6a5492782 P 4da400afe85b441096ee22a8d226ed68 [CANDIDATE]: Term 1 election: Requested vote from peers 65e675b6f229423f836afdbfed785fd4 (127.4.202.131:34281), acfa18225c5b4a5ca3ea16defd40c0b6 (127.4.202.130:35917)
I20260926 06:20:15.737150 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bccb10ed592c4b5582e1e5a6a5492782" candidate_uuid: "4da400afe85b441096ee22a8d226ed68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65e675b6f229423f836afdbfed785fd4"
I20260926 06:20:15.737221 9416 raft_consensus.cc:3037] T bccb10ed592c4b5582e1e5a6a5492782 P 65e675b6f229423f836afdbfed785fd4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:15.737823 9416 raft_consensus.cc:2454] T bccb10ed592c4b5582e1e5a6a5492782 P 65e675b6f229423f836afdbfed785fd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4da400afe85b441096ee22a8d226ed68 in term 1.
I20260926 06:20:15.737978 9084 leader_election.cc:302] T bccb10ed592c4b5582e1e5a6a5492782 P 4da400afe85b441096ee22a8d226ed68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4da400afe85b441096ee22a8d226ed68, 65e675b6f229423f836afdbfed785fd4; no voters:
I20260926 06:20:15.738055 9475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:15.732716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:58977 (local address 127.4.202.130:35917)
0926 06:20:15.732750 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:15.732751 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:15.735000 (+ 2249us) server_negotiation.cc:407] Connection header received
0926 06:20:15.735098 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:15.735099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:15.735108 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:15.735159 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:20:15.735283 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.735932 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.737296 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.737711 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.737813 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:15.737822 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:15.737971 (+ 149us) server_negotiation.cc:299] Negotiation successful
0926 06:20:15.738003 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 06:20:15.738062 9482 raft_consensus.cc:2781] T bccb10ed592c4b5582e1e5a6a5492782 P 4da400afe85b441096ee22a8d226ed68 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:20:15.738075 9485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:15.732574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:35917 (local address 127.4.202.129:58977)
0926 06:20:15.734940 (+ 2366us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:15.734950 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:15.735022 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:15.735163 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:15.735163 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:15.735165 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:20:15.735224 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:15.735225 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.736007 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.736008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:15.737188 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:15.737190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.737832 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.737833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:15.737907 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:15.737910 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:20:15.737965 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 06:20:15.737999 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2350,"thread_start_us":66,"threads_started":1}
I20260926 06:20:15.738154 9482 raft_consensus.cc:686] T bccb10ed592c4b5582e1e5a6a5492782 P 4da400afe85b441096ee22a8d226ed68 [term 1 LEADER]: Becoming Leader. State: Replica: 4da400afe85b441096ee22a8d226ed68, State: Running, Role: LEADER
I20260926 06:20:15.738230 9482 consensus_queue.cc:237] T bccb10ed592c4b5582e1e5a6a5492782 P 4da400afe85b441096ee22a8d226ed68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65e675b6f229423f836afdbfed785fd4" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 34281 } } peers { permanent_uuid: "4da400afe85b441096ee22a8d226ed68" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37229 } } peers { permanent_uuid: "acfa18225c5b4a5ca3ea16defd40c0b6" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 35917 } }
I20260926 06:20:15.738379 9282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bccb10ed592c4b5582e1e5a6a5492782" candidate_uuid: "4da400afe85b441096ee22a8d226ed68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfa18225c5b4a5ca3ea16defd40c0b6"
I20260926 06:20:15.738466 9282 raft_consensus.cc:3037] T bccb10ed592c4b5582e1e5a6a5492782 P acfa18225c5b4a5ca3ea16defd40c0b6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:15.738360 9283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bccb10ed592c4b5582e1e5a6a5492782" candidate_uuid: "4da400afe85b441096ee22a8d226ed68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfa18225c5b4a5ca3ea16defd40c0b6" is_pre_election: true
I20260926 06:20:15.739280 9282 raft_consensus.cc:2454] T bccb10ed592c4b5582e1e5a6a5492782 P acfa18225c5b4a5ca3ea16defd40c0b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4da400afe85b441096ee22a8d226ed68 in term 1.
I20260926 06:20:15.739516 9006 catalog_manager.cc:5495] T bccb10ed592c4b5582e1e5a6a5492782 P 4da400afe85b441096ee22a8d226ed68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4da400afe85b441096ee22a8d226ed68 (127.4.202.129). New cstate: current_term: 1 leader_uuid: "4da400afe85b441096ee22a8d226ed68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65e675b6f229423f836afdbfed785fd4" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 34281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4da400afe85b441096ee22a8d226ed68" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acfa18225c5b4a5ca3ea16defd40c0b6" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 35917 } health_report { overall_health: UNKNOWN } } }
I20260926 06:20:15.748987 9474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:15.747027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58692 (local address 127.4.202.129:37229)
0926 06:20:15.747058 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:15.747062 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:15.747110 (+ 48us) server_negotiation.cc:407] Connection header received
0926 06:20:15.747162 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:15.747163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:15.747175 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:15.747229 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:20:15.747351 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.747756 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.748415 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:15.748588 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:15.748632 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:15.748775 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:20:15.748849 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:20:15.748902 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:15.748934 (+ 32us) server_negotiation.cc:299] Negotiation successful
0926 06:20:15.748948 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 06:20:15.750326 9282 raft_consensus.cc:1261] T bccb10ed592c4b5582e1e5a6a5492782 P acfa18225c5b4a5ca3ea16defd40c0b6 [term 1 FOLLOWER]: Refusing update from remote peer 4da400afe85b441096ee22a8d226ed68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:20:15.750375 9416 raft_consensus.cc:1261] T bccb10ed592c4b5582e1e5a6a5492782 P 65e675b6f229423f836afdbfed785fd4 [term 1 FOLLOWER]: Refusing update from remote peer 4da400afe85b441096ee22a8d226ed68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:20:15.750672 9484 consensus_queue.cc:1035] T bccb10ed592c4b5582e1e5a6a5492782 P 4da400afe85b441096ee22a8d226ed68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfa18225c5b4a5ca3ea16defd40c0b6" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 35917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:20:15.750814 9482 consensus_queue.cc:1035] T bccb10ed592c4b5582e1e5a6a5492782 P 4da400afe85b441096ee22a8d226ed68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65e675b6f229423f836afdbfed785fd4" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 34281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:20:15.752696 9489 mvcc.cc:205] Tried to move back new op lower bound from 7333493210111389696 to 7333493210064416768. Current Snapshot: MvccSnapshot[applied={T|T < 7333493210111389696}]
I20260926 06:20:15.753222 9490 mvcc.cc:205] Tried to move back new op lower bound from 7333493210111389696 to 7333493210064416768. Current Snapshot: MvccSnapshot[applied={T|T < 7333493210111389696}]
I20260926 06:20:15.753232 9491 mvcc.cc:205] Tried to move back new op lower bound from 7333493210111389696 to 7333493210064416768. Current Snapshot: MvccSnapshot[applied={T|T < 7333493210111389696}]
I20260926 06:20:15.754884 9004 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:20:15.754971 9004 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:20:15.755582 9004 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2540a8b227dc410292c1802cdbc7cfdb: Sending DeleteTablet for 3 replicas of tablet bccb10ed592c4b5582e1e5a6a5492782
I20260926 06:20:15.755884 9396 tablet_service.cc:1514] Processing DeleteTablet for tablet bccb10ed592c4b5582e1e5a6a5492782 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:20:15 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:37604
I20260926 06:20:15.756090 9499 tablet_replica.cc:330] T bccb10ed592c4b5582e1e5a6a5492782 P 65e675b6f229423f836afdbfed785fd4: stopping tablet replica
I20260926 06:20:15.756105 9263 tablet_service.cc:1514] Processing DeleteTablet for tablet bccb10ed592c4b5582e1e5a6a5492782 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:20:15 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:53838
I20260926 06:20:15.756166 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 9068
I20260926 06:20:15.756209 9499 raft_consensus.cc:2229] T bccb10ed592c4b5582e1e5a6a5492782 P 65e675b6f229423f836afdbfed785fd4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:20:15.756295 9499 raft_consensus.cc:2258] T bccb10ed592c4b5582e1e5a6a5492782 P 65e675b6f229423f836afdbfed785fd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:20:15.757051 9499 ts_tablet_manager.cc:1905] T bccb10ed592c4b5582e1e5a6a5492782 P 65e675b6f229423f836afdbfed785fd4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:20:15.757123 9500 tablet_replica.cc:330] T bccb10ed592c4b5582e1e5a6a5492782 P acfa18225c5b4a5ca3ea16defd40c0b6: stopping tablet replica
I20260926 06:20:15.757212 9500 raft_consensus.cc:2229] T bccb10ed592c4b5582e1e5a6a5492782 P acfa18225c5b4a5ca3ea16defd40c0b6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:20:15.757301 9500 raft_consensus.cc:2258] T bccb10ed592c4b5582e1e5a6a5492782 P acfa18225c5b4a5ca3ea16defd40c0b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:20:15.757654 9500 ts_tablet_manager.cc:1905] T bccb10ed592c4b5582e1e5a6a5492782 P acfa18225c5b4a5ca3ea16defd40c0b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:20:15.758924 9499 ts_tablet_manager.cc:1918] T bccb10ed592c4b5582e1e5a6a5492782 P 65e675b6f229423f836afdbfed785fd4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:20:15.758957 9500 ts_tablet_manager.cc:1918] T bccb10ed592c4b5582e1e5a6a5492782 P acfa18225c5b4a5ca3ea16defd40c0b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:20:15.759018 9500 log.cc:1196] T bccb10ed592c4b5582e1e5a6a5492782 P acfa18225c5b4a5ca3ea16defd40c0b6: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-1/wal/wals/bccb10ed592c4b5582e1e5a6a5492782
I20260926 06:20:15.759037 9499 log.cc:1196] T bccb10ed592c4b5582e1e5a6a5492782 P 65e675b6f229423f836afdbfed785fd4: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790403591640726-4906-0/minicluster-data/ts-2/wal/wals/bccb10ed592c4b5582e1e5a6a5492782
I20260926 06:20:15.759307 9499 ts_tablet_manager.cc:1939] T bccb10ed592c4b5582e1e5a6a5492782 P 65e675b6f229423f836afdbfed785fd4: Deleting consensus metadata
I20260926 06:20:15.759321 9500 ts_tablet_manager.cc:1939] T bccb10ed592c4b5582e1e5a6a5492782 P acfa18225c5b4a5ca3ea16defd40c0b6: Deleting consensus metadata
I20260926 06:20:15.759687 8991 catalog_manager.cc:4841] TS 65e675b6f229423f836afdbfed785fd4 (127.4.202.131:34281): tablet bccb10ed592c4b5582e1e5a6a5492782 (table test-table [id=c22fcdea047f492086da7a5738f404e4]) successfully deleted
I20260926 06:20:15.759819 8993 catalog_manager.cc:4841] TS acfa18225c5b4a5ca3ea16defd40c0b6 (127.4.202.130:35917): tablet bccb10ed592c4b5582e1e5a6a5492782 (table test-table [id=c22fcdea047f492086da7a5738f404e4]) successfully deleted
W20260926 06:20:15.763111 8992 connection.cc:535] client connection to 127.4.202.129:37229 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 06:20:15.763228 8992 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 06:20:15.763247 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 9201
I20260926 06:20:15.767021 9471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:15.766906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:37229 (local address 127.0.0.1:58702)
0926 06:20:15.766959 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:15.766997 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.129:37229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260926 06:20:15.767164 8992 catalog_manager.cc:4584] TS 4da400afe85b441096ee22a8d226ed68 (127.4.202.129:37229): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bccb10ed592c4b5582e1e5a6a5492782: Network error: Client connection negotiation failed: client connection to 127.4.202.129:37229: connect: Connection refused (error 111)
I20260926 06:20:15.768276 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 9334
I20260926 06:20:15.772629 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 8975
2026-09-26T06:20:15Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4981 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-26T06:20:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:20:15Z Disabled control of system clock
I20260926 06:20:15.798424 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:36425
--webserver_interface=127.4.202.190
--webserver_port=0
--builtin_ntp_servers=127.4.202.148:43609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:36425
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 06:20:15.876147 9505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:15.876334 9505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:15.876355 9505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:15.877466 9505 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:20:15.877504 9505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:15.877516 9505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:15.877529 9505 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:20:15.877564 9505 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:20:15.878893 9505 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:43609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:36425
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:36425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:15.879145 9505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:15.879468 9505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:15.884217 9510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:15.884215 9511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:15.884413 9505 server_base.cc:1034] running on GCE node
W20260926 06:20:15.884210 9513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:15.884691 9505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:15.884960 9505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:15.886168 9505 hybrid_clock.cc:648] HybridClock initialized: now 1790403615886150 us; error 65 us; skew 500 ppm
I20260926 06:20:15.887492 9505 webserver.cc:460] Webserver started at http://127.4.202.190:44743/ using document root <none> and password file <none>
I20260926 06:20:15.887759 9505 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:15.887809 9505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:15.887951 9505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:15.888949 9505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "5ca1cffe650344dd83a3a33c537babe9"
format_stamp: "Formatted at 2026-09-26 06:20:15 on dist-test-slave-pd0t"
server_key: "6dc8715089e152fb4caa968583ec3d38"
server_key_iv: "b21aec41f90660b20c3fabd02e993ebb"
server_key_version: "encryptionkey@0"
I20260926 06:20:15.889295 9505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "5ca1cffe650344dd83a3a33c537babe9"
format_stamp: "Formatted at 2026-09-26 06:20:15 on dist-test-slave-pd0t"
server_key: "6dc8715089e152fb4caa968583ec3d38"
server_key_iv: "b21aec41f90660b20c3fabd02e993ebb"
server_key_version: "encryptionkey@0"
I20260926 06:20:15.890532 9505 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:20:15.891237 9519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:15.891427 9505 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:20:15.891515 9505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "5ca1cffe650344dd83a3a33c537babe9"
format_stamp: "Formatted at 2026-09-26 06:20:15 on dist-test-slave-pd0t"
server_key: "6dc8715089e152fb4caa968583ec3d38"
server_key_iv: "b21aec41f90660b20c3fabd02e993ebb"
server_key_version: "encryptionkey@0"
I20260926 06:20:15.891580 9505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:15.894827 9505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:15.895066 9505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:15.895192 9505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:15.898726 9505 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:36425
I20260926 06:20:15.898752 9571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:36425 every 8 connection(s)
I20260926 06:20:15.899050 9505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:20:15.899617 9572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:15.902019 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 9505
I20260926 06:20:15.902101 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:20:15.902134 9572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ca1cffe650344dd83a3a33c537babe9: Bootstrap starting.
I20260926 06:20:15.902314 4906 external_mini_cluster.cc:1428] Setting key 47e25b7aa3cb78d16680bcafa9c61712
I20260926 06:20:15.902856 9572 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ca1cffe650344dd83a3a33c537babe9: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:15.903143 9572 log.cc:824] T 00000000000000000000000000000000 P 5ca1cffe650344dd83a3a33c537babe9: Log is configured to *not* fsync() on all Append() calls
I20260926 06:20:15.903802 9572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ca1cffe650344dd83a3a33c537babe9: No bootstrap required, opened a new log
I20260926 06:20:15.904663 9575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:15.902820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34666 (local address 127.4.202.190:36425)
0926 06:20:15.903009 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:15.903017 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:15.903043 (+ 26us) server_negotiation.cc:407] Connection header received
0926 06:20:15.903516 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:15.903523 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:15.903558 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:15.903698 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:15.904267 (+ 569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:15.904270 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:15.904270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:20:15.904275 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:15.904324 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:20:15.904333 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:15.904426 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:15.904467 (+ 41us) server_negotiation.cc:299] Negotiation successful
0926 06:20:15.904520 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":71,"threads_started":1}
I20260926 06:20:15.905623 9572 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ca1cffe650344dd83a3a33c537babe9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ca1cffe650344dd83a3a33c537babe9" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 36425 } }
I20260926 06:20:15.905766 9572 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ca1cffe650344dd83a3a33c537babe9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:15.905812 9572 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ca1cffe650344dd83a3a33c537babe9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ca1cffe650344dd83a3a33c537babe9, State: Initialized, Role: FOLLOWER
I20260926 06:20:15.905954 9572 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ca1cffe650344dd83a3a33c537babe9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ca1cffe650344dd83a3a33c537babe9" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 36425 } }
I20260926 06:20:15.906052 9572 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ca1cffe650344dd83a3a33c537babe9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:20:15.906096 9572 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ca1cffe650344dd83a3a33c537babe9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:20:15.906143 9572 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ca1cffe650344dd83a3a33c537babe9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:15.907015 9572 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ca1cffe650344dd83a3a33c537babe9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ca1cffe650344dd83a3a33c537babe9" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 36425 } }
I20260926 06:20:15.907128 9572 leader_election.cc:302] T 00000000000000000000000000000000 P 5ca1cffe650344dd83a3a33c537babe9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ca1cffe650344dd83a3a33c537babe9; no voters:
I20260926 06:20:15.907297 9572 leader_election.cc:288] T 00000000000000000000000000000000 P 5ca1cffe650344dd83a3a33c537babe9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:20:15.907456 9577 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ca1cffe650344dd83a3a33c537babe9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:20:15.907539 9572 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ca1cffe650344dd83a3a33c537babe9 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:20:15.907605 9577 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ca1cffe650344dd83a3a33c537babe9 [term 1 LEADER]: Becoming Leader. State: Replica: 5ca1cffe650344dd83a3a33c537babe9, State: Running, Role: LEADER
I20260926 06:20:15.907915 9577 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ca1cffe650344dd83a3a33c537babe9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ca1cffe650344dd83a3a33c537babe9" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 36425 } }
I20260926 06:20:15.908301 9578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ca1cffe650344dd83a3a33c537babe9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ca1cffe650344dd83a3a33c537babe9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ca1cffe650344dd83a3a33c537babe9" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 36425 } } }
I20260926 06:20:15.908298 9579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ca1cffe650344dd83a3a33c537babe9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ca1cffe650344dd83a3a33c537babe9. Latest consensus state: current_term: 1 leader_uuid: "5ca1cffe650344dd83a3a33c537babe9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ca1cffe650344dd83a3a33c537babe9" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 36425 } } }
I20260926 06:20:15.908407 9578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ca1cffe650344dd83a3a33c537babe9 [sys.catalog]: This master's current role is: LEADER
I20260926 06:20:15.908430 9579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ca1cffe650344dd83a3a33c537babe9 [sys.catalog]: This master's current role is: LEADER
I20260926 06:20:15.908600 9586 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:20:15.909070 9586 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:20:15.910499 9586 catalog_manager.cc:1347] Generated new cluster ID: 586f2cc3c455462c95c8f70a48534136
I20260926 06:20:15.910555 9586 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:20:15.922130 9586 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:20:15.922286 9586 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:20:15.925998 9586 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ca1cffe650344dd83a3a33c537babe9: Generated new TSK 0
I20260926 06:20:15.926148 9586 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:20:15.934832 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.129:0
--local_ip_for_outbound_sockets=127.4.202.129
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:36425
--builtin_ntp_servers=127.4.202.148:43609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 06:20:16.020946 9596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:16.021152 9596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:16.021188 9596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:16.022310 9596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:16.022372 9596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:16.022416 9596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.129
I20260926 06:20:16.023784 9596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:43609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:36425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.129
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:16.024040 9596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:16.024308 9596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:16.029950 9604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:16.030014 9602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:16.029963 9601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:16.030236 9596 server_base.cc:1034] running on GCE node
I20260926 06:20:16.030493 9596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:16.030777 9596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:16.031984 9596 hybrid_clock.cc:648] HybridClock initialized: now 1790403616031971 us; error 27 us; skew 500 ppm
I20260926 06:20:16.033417 9596 webserver.cc:460] Webserver started at http://127.4.202.129:43707/ using document root <none> and password file <none>
I20260926 06:20:16.033742 9596 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:16.033805 9596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:16.033923 9596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:16.035384 9596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-0/data/instance:
uuid: "72d08db2f2f440c6a153098d610864fb"
format_stamp: "Formatted at 2026-09-26 06:20:16 on dist-test-slave-pd0t"
server_key: "dc067883364eadf1af3c57c885807aae"
server_key_iv: "918ad72fcc9e28102c3ced75b1e61026"
server_key_version: "encryptionkey@0"
I20260926 06:20:16.035805 9596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance:
uuid: "72d08db2f2f440c6a153098d610864fb"
format_stamp: "Formatted at 2026-09-26 06:20:16 on dist-test-slave-pd0t"
server_key: "dc067883364eadf1af3c57c885807aae"
server_key_iv: "918ad72fcc9e28102c3ced75b1e61026"
server_key_version: "encryptionkey@0"
I20260926 06:20:16.037343 9596 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:20:16.038127 9610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:16.038328 9596 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:20:16.038412 9596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-0/wal
uuid: "72d08db2f2f440c6a153098d610864fb"
format_stamp: "Formatted at 2026-09-26 06:20:16 on dist-test-slave-pd0t"
server_key: "dc067883364eadf1af3c57c885807aae"
server_key_iv: "918ad72fcc9e28102c3ced75b1e61026"
server_key_version: "encryptionkey@0"
I20260926 06:20:16.038468 9596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:16.042896 9596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:16.043175 9596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:16.043315 9596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:16.043516 9596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:20:16.043835 9596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:20:16.043875 9596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:16.043900 9596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:20:16.043915 9596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:16.050190 9596 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.129:42863
I20260926 06:20:16.050246 9723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.129:42863 every 8 connection(s)
I20260926 06:20:16.050518 9596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
I20260926 06:20:16.051170 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 9596
I20260926 06:20:16.051295 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance
I20260926 06:20:16.051507 4906 external_mini_cluster.cc:1428] Setting key f62c52a91c6487db85167de2afaa5084
I20260926 06:20:16.052376 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.130:0
--local_ip_for_outbound_sockets=127.4.202.130
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:36425
--builtin_ntp_servers=127.4.202.148:43609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260926 06:20:16.053953 9575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:16.051587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:33277 (local address 127.4.202.190:36425)
0926 06:20:16.051636 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:16.051638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:16.051669 (+ 31us) server_negotiation.cc:407] Connection header received
0926 06:20:16.052277 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:16.052278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:16.052285 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:16.052356 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:16.053720 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:16.053721 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:16.053721 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:20:16.053726 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:16.053740 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:20:16.053744 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:16.053795 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:16.053892 (+ 97us) server_negotiation.cc:299] Negotiation successful
0926 06:20:16.053904 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 06:20:16.054178 9726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:16.051342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:36425 (local address 127.4.202.129:33277)
0926 06:20:16.051536 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:16.051593 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:16.052165 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:16.052433 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:16.052438 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:16.052449 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:16.053546 (+ 1097us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:20:16.053547 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:16.053585 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:20:16.053621 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:16.053813 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:16.053816 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:16.053825 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 06:20:16.053897 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 06:20:16.053958 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":103,"threads_started":1}
I20260926 06:20:16.054678 9724 heartbeater.cc:347] Connected to a master server at 127.4.202.190:36425
I20260926 06:20:16.054766 9724 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:16.054908 9724 heartbeater.cc:511] Master 127.4.202.190:36425 requested a full tablet report, sending...
I20260926 06:20:16.055327 9536 ts_manager.cc:194] Registered new tserver with Master: 72d08db2f2f440c6a153098d610864fb (127.4.202.129:42863)
W20260926 06:20:16.131821 9727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:16.132002 9727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:16.132021 9727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:16.133175 9727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:16.133214 9727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:16.133245 9727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.130
I20260926 06:20:16.134598 9727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:43609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:36425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.130
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:16.134817 9727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:16.135099 9727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:16.139904 9732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:16.139983 9735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:16.140054 9727 server_base.cc:1034] running on GCE node
W20260926 06:20:16.140192 9733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:16.140442 9727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:16.140673 9727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:16.141861 9727 hybrid_clock.cc:648] HybridClock initialized: now 1790403616141858 us; error 38 us; skew 500 ppm
I20260926 06:20:16.143167 9727 webserver.cc:460] Webserver started at http://127.4.202.130:39423/ using document root <none> and password file <none>
I20260926 06:20:16.143426 9727 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:16.143498 9727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:16.143688 9727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:16.144673 9727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-1/data/instance:
uuid: "5182268877c348b7a48caffc5a6c695e"
format_stamp: "Formatted at 2026-09-26 06:20:16 on dist-test-slave-pd0t"
server_key: "12c573d1eb4a547a06a1569471a4a2ac"
server_key_iv: "7abc713623efc8adaabc4a74e2672f0e"
server_key_version: "encryptionkey@0"
I20260926 06:20:16.145026 9727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance:
uuid: "5182268877c348b7a48caffc5a6c695e"
format_stamp: "Formatted at 2026-09-26 06:20:16 on dist-test-slave-pd0t"
server_key: "12c573d1eb4a547a06a1569471a4a2ac"
server_key_iv: "7abc713623efc8adaabc4a74e2672f0e"
server_key_version: "encryptionkey@0"
I20260926 06:20:16.146456 9727 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 06:20:16.147435 9741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:16.147589 9727 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:20:16.147687 9727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-1/wal
uuid: "5182268877c348b7a48caffc5a6c695e"
format_stamp: "Formatted at 2026-09-26 06:20:16 on dist-test-slave-pd0t"
server_key: "12c573d1eb4a547a06a1569471a4a2ac"
server_key_iv: "7abc713623efc8adaabc4a74e2672f0e"
server_key_version: "encryptionkey@0"
I20260926 06:20:16.147765 9727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:16.150907 9727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:16.151151 9727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:16.151294 9727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:16.151489 9727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:20:16.151767 9727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:20:16.151798 9727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:16.151851 9727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:20:16.151880 9727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:16.157342 9727 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.130:41233
I20260926 06:20:16.157415 9854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.130:41233 every 8 connection(s)
I20260926 06:20:16.157692 9727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
I20260926 06:20:16.160561 9575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:16.158372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:53651 (local address 127.4.202.190:36425)
0926 06:20:16.158444 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:16.158446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:16.158660 (+ 214us) server_negotiation.cc:407] Connection header received
0926 06:20:16.159137 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:16.159139 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:16.159146 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:16.159196 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:16.160307 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:16.160309 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:16.160309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:20:16.160313 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:16.160329 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:20:16.160333 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:16.160389 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:16.160479 (+ 90us) server_negotiation.cc:299] Negotiation successful
0926 06:20:16.160494 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 06:20:16.160756 9857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:16.158350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:36425 (local address 127.4.202.130:53651)
0926 06:20:16.158529 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:16.158559 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:16.159025 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:16.159259 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:16.159263 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:16.159273 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:16.160124 (+ 851us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:20:16.160125 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:16.160150 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:20:16.160175 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:16.160400 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:16.160404 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:16.160407 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:20:16.160490 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 06:20:16.160544 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":77,"threads_started":1}
I20260926 06:20:16.161208 9855 heartbeater.cc:347] Connected to a master server at 127.4.202.190:36425
I20260926 06:20:16.161299 9855 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:16.161407 9855 heartbeater.cc:511] Master 127.4.202.190:36425 requested a full tablet report, sending...
I20260926 06:20:16.161701 9536 ts_manager.cc:194] Registered new tserver with Master: 5182268877c348b7a48caffc5a6c695e (127.4.202.130:41233)
I20260926 06:20:16.167158 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 9727
I20260926 06:20:16.167232 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance
I20260926 06:20:16.167371 4906 external_mini_cluster.cc:1428] Setting key 38ef59fbc1607e502c8b7cbe5b8e8886
I20260926 06:20:16.168187 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.131:0
--local_ip_for_outbound_sockets=127.4.202.131
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:36425
--builtin_ntp_servers=127.4.202.148:43609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 06:20:16.248659 9858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:16.248875 9858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:16.248912 9858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:16.250219 9858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:16.250278 9858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:16.250322 9858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.131
I20260926 06:20:16.251760 9858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:43609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:36425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.131
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:16.251993 9858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:16.252281 9858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:16.256966 9864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:16.256958 9866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:16.256973 9863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:16.257076 9858 server_base.cc:1034] running on GCE node
I20260926 06:20:16.257314 9858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:16.257570 9858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:16.258744 9858 hybrid_clock.cc:648] HybridClock initialized: now 1790403616258717 us; error 38 us; skew 500 ppm
I20260926 06:20:16.259974 9858 webserver.cc:460] Webserver started at http://127.4.202.131:38481/ using document root <none> and password file <none>
I20260926 06:20:16.260239 9858 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:16.260318 9858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:16.260483 9858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:16.261440 9858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-2/data/instance:
uuid: "7380b48f51f948d58ba12ee3f695ab3e"
format_stamp: "Formatted at 2026-09-26 06:20:16 on dist-test-slave-pd0t"
server_key: "fd318ffff8205d4bc430d6e2bb5f0938"
server_key_iv: "1cba382ed91a499af5d0fb86eb6d1098"
server_key_version: "encryptionkey@0"
I20260926 06:20:16.261790 9858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance:
uuid: "7380b48f51f948d58ba12ee3f695ab3e"
format_stamp: "Formatted at 2026-09-26 06:20:16 on dist-test-slave-pd0t"
server_key: "fd318ffff8205d4bc430d6e2bb5f0938"
server_key_iv: "1cba382ed91a499af5d0fb86eb6d1098"
server_key_version: "encryptionkey@0"
I20260926 06:20:16.263006 9858 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:20:16.263672 9872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:16.263860 9858 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:20:16.263952 9858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-2/wal
uuid: "7380b48f51f948d58ba12ee3f695ab3e"
format_stamp: "Formatted at 2026-09-26 06:20:16 on dist-test-slave-pd0t"
server_key: "fd318ffff8205d4bc430d6e2bb5f0938"
server_key_iv: "1cba382ed91a499af5d0fb86eb6d1098"
server_key_version: "encryptionkey@0"
I20260926 06:20:16.264031 9858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:16.267171 9858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:16.267400 9858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:16.267532 9858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:16.267731 9858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:20:16.268004 9858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:20:16.268054 9858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:16.268090 9858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:20:16.268116 9858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:16.273591 9858 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.131:38325
I20260926 06:20:16.273658 9985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.131:38325 every 8 connection(s)
I20260926 06:20:16.273963 9858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
I20260926 06:20:16.276919 9575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:16.274632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:55285 (local address 127.4.202.190:36425)
0926 06:20:16.274815 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:16.274818 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:16.275024 (+ 206us) server_negotiation.cc:407] Connection header received
0926 06:20:16.275482 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:16.275483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:16.275489 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:16.275544 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:16.276660 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:16.276662 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:16.276662 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:20:16.276667 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:16.276681 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:20:16.276685 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:16.276735 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:16.276829 (+ 94us) server_negotiation.cc:299] Negotiation successful
0926 06:20:16.276845 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 06:20:16.277078 9988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:16.274611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:36425 (local address 127.4.202.131:55285)
0926 06:20:16.274916 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:16.274945 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:16.275377 (+ 432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:16.275608 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:16.275612 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:16.275621 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:16.276517 (+ 896us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:20:16.276517 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:16.276541 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:20:16.276565 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:16.276732 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:16.276736 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:16.276739 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:20:16.276829 (+ 90us) client_negotiation.cc:240] Negotiation successful
0926 06:20:16.276884 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":77,"threads_started":1}
I20260926 06:20:16.277549 9986 heartbeater.cc:347] Connected to a master server at 127.4.202.190:36425
I20260926 06:20:16.277638 9986 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:16.277766 9986 heartbeater.cc:511] Master 127.4.202.190:36425 requested a full tablet report, sending...
I20260926 06:20:16.278133 9536 ts_manager.cc:194] Registered new tserver with Master: 7380b48f51f948d58ba12ee3f695ab3e (127.4.202.131:38325)
I20260926 06:20:16.282290 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 9858
I20260926 06:20:16.282367 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance
I20260926 06:20:16.282497 4906 external_mini_cluster.cc:1428] Setting key d71ba5d5d20a7761ee1afcc891752312
I20260926 06:20:16.283346 4906 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:20:16.285535 9575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:16.284584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34674 (local address 127.4.202.190:36425)
0926 06:20:16.284648 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:16.284650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:16.284703 (+ 53us) server_negotiation.cc:407] Connection header received
0926 06:20:16.284743 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:16.284744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:16.284747 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:16.284793 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:16.285344 (+ 551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:16.285345 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:16.285345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:20:16.285348 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:16.285356 (+ 8us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:20:16.285359 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:16.285398 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:16.285429 (+ 31us) server_negotiation.cc:299] Negotiation successful
0926 06:20:16.285439 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 06:20:16.286969 9536 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34674:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:20:16.288389 9536 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:20:16.291718 9995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:16.290343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:42863 (local address 127.0.0.1:41134)
0926 06:20:16.290552 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:16.290562 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:16.290624 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:16.290869 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:16.290874 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:16.290880 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:16.291320 (+ 440us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:20:16.291320 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:16.291333 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:20:16.291351 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:16.291573 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:16.291577 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:16.291580 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:20:16.291643 (+ 63us) client_negotiation.cc:240] Negotiation successful
0926 06:20:16.291656 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":87,"threads_started":1}
I20260926 06:20:16.291862 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:16.290298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:41233 (local address 127.0.0.1:51940)
0926 06:20:16.290443 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:16.290477 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:16.290530 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:16.290968 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:16.290969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:16.290972 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:16.291471 (+ 499us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:20:16.291472 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:16.291477 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:20:16.291488 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:16.291727 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:16.291728 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:16.291729 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 06:20:16.291760 (+ 31us) client_negotiation.cc:240] Negotiation successful
0926 06:20:16.291776 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":47,"threads_started":1}
I20260926 06:20:16.291956 9998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:16.290550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41134 (local address 127.4.202.129:42863)
0926 06:20:16.290749 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:16.290755 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:16.290764 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:20:16.290790 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:16.290792 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:16.290802 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:16.290869 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:16.291432 (+ 563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:16.291438 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:16.291438 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:20:16.291443 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:16.291476 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:20:16.291497 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:16.291550 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:16.291665 (+ 115us) server_negotiation.cc:299] Negotiation successful
0926 06:20:16.291680 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":101,"threads_started":1}
I20260926 06:20:16.292111 9997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:16.290379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51940 (local address 127.4.202.130:41233)
0926 06:20:16.290821 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:16.290825 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:16.290834 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:20:16.290858 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:16.290860 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:16.290868 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:16.291114 (+ 246us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:16.291567 (+ 453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:16.291576 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:16.291577 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:20:16.291581 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:16.291614 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:20:16.291642 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:16.291976 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:16.292004 (+ 28us) server_negotiation.cc:299] Negotiation successful
0926 06:20:16.292022 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":82,"threads_started":1}
I20260926 06:20:16.292522 9996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:16.290378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:38325 (local address 127.0.0.1:40846)
0926 06:20:16.290731 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:16.290739 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:16.290771 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:16.291485 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:16.291486 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:16.291488 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:16.292165 (+ 677us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:20:16.292166 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:16.292173 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:20:16.292200 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:16.292391 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:16.292392 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:16.292393 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 06:20:16.292452 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 06:20:16.292464 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":54,"threads_started":1}
I20260926 06:20:16.292572 9999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:16.290657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40846 (local address 127.4.202.131:38325)
0926 06:20:16.290781 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:16.290785 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:16.290796 (+ 11us) server_negotiation.cc:407] Connection header received
0926 06:20:16.290819 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:16.290821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:16.290830 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:16.290910 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:16.292260 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:16.292267 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:16.292267 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:20:16.292271 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:16.292297 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:20:16.292314 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:16.292396 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:16.292483 (+ 87us) server_negotiation.cc:299] Negotiation successful
0926 06:20:16.292498 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":61,"threads_started":1}
I20260926 06:20:16.292578 9658 tablet_service.cc:1467] Processing CreateTablet for tablet 5c55ffa5efdd4c78a44aa2addcceb183 (DEFAULT_TABLE table=test-table [id=53ab13e314e740f49e783d4ceadaad81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:20:16.292682 9789 tablet_service.cc:1467] Processing CreateTablet for tablet 5c55ffa5efdd4c78a44aa2addcceb183 (DEFAULT_TABLE table=test-table [id=53ab13e314e740f49e783d4ceadaad81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:20:16.292935 9658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c55ffa5efdd4c78a44aa2addcceb183. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:16.292961 9789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c55ffa5efdd4c78a44aa2addcceb183. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:16.293107 9920 tablet_service.cc:1467] Processing CreateTablet for tablet 5c55ffa5efdd4c78a44aa2addcceb183 (DEFAULT_TABLE table=test-table [id=53ab13e314e740f49e783d4ceadaad81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:20:16.293324 9920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c55ffa5efdd4c78a44aa2addcceb183. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:16.295936 10000 tablet_bootstrap.cc:492] T 5c55ffa5efdd4c78a44aa2addcceb183 P 7380b48f51f948d58ba12ee3f695ab3e: Bootstrap starting.
I20260926 06:20:16.296494 10000 tablet_bootstrap.cc:654] T 5c55ffa5efdd4c78a44aa2addcceb183 P 7380b48f51f948d58ba12ee3f695ab3e: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:16.296590 10002 tablet_bootstrap.cc:492] T 5c55ffa5efdd4c78a44aa2addcceb183 P 5182268877c348b7a48caffc5a6c695e: Bootstrap starting.
I20260926 06:20:16.296772 10001 tablet_bootstrap.cc:492] T 5c55ffa5efdd4c78a44aa2addcceb183 P 72d08db2f2f440c6a153098d610864fb: Bootstrap starting.
I20260926 06:20:16.296795 10000 log.cc:824] T 5c55ffa5efdd4c78a44aa2addcceb183 P 7380b48f51f948d58ba12ee3f695ab3e: Log is configured to *not* fsync() on all Append() calls
I20260926 06:20:16.297232 10002 tablet_bootstrap.cc:654] T 5c55ffa5efdd4c78a44aa2addcceb183 P 5182268877c348b7a48caffc5a6c695e: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:16.297390 10001 tablet_bootstrap.cc:654] T 5c55ffa5efdd4c78a44aa2addcceb183 P 72d08db2f2f440c6a153098d610864fb: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:16.297556 10002 log.cc:824] T 5c55ffa5efdd4c78a44aa2addcceb183 P 5182268877c348b7a48caffc5a6c695e: Log is configured to *not* fsync() on all Append() calls
I20260926 06:20:16.297679 10001 log.cc:824] T 5c55ffa5efdd4c78a44aa2addcceb183 P 72d08db2f2f440c6a153098d610864fb: Log is configured to *not* fsync() on all Append() calls
I20260926 06:20:16.298283 10002 tablet_bootstrap.cc:492] T 5c55ffa5efdd4c78a44aa2addcceb183 P 5182268877c348b7a48caffc5a6c695e: No bootstrap required, opened a new log
I20260926 06:20:16.298280 10000 tablet_bootstrap.cc:492] T 5c55ffa5efdd4c78a44aa2addcceb183 P 7380b48f51f948d58ba12ee3f695ab3e: No bootstrap required, opened a new log
I20260926 06:20:16.298293 10001 tablet_bootstrap.cc:492] T 5c55ffa5efdd4c78a44aa2addcceb183 P 72d08db2f2f440c6a153098d610864fb: No bootstrap required, opened a new log
I20260926 06:20:16.298341 10000 ts_tablet_manager.cc:1397] T 5c55ffa5efdd4c78a44aa2addcceb183 P 7380b48f51f948d58ba12ee3f695ab3e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:20:16.298341 10001 ts_tablet_manager.cc:1397] T 5c55ffa5efdd4c78a44aa2addcceb183 P 72d08db2f2f440c6a153098d610864fb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:20:16.298363 10002 ts_tablet_manager.cc:1397] T 5c55ffa5efdd4c78a44aa2addcceb183 P 5182268877c348b7a48caffc5a6c695e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:20:16.300277 10002 raft_consensus.cc:348] T 5c55ffa5efdd4c78a44aa2addcceb183 P 5182268877c348b7a48caffc5a6c695e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5182268877c348b7a48caffc5a6c695e" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 41233 } } peers { permanent_uuid: "72d08db2f2f440c6a153098d610864fb" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 42863 } } peers { permanent_uuid: "7380b48f51f948d58ba12ee3f695ab3e" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 38325 } }
I20260926 06:20:16.300427 10002 raft_consensus.cc:374] T 5c55ffa5efdd4c78a44aa2addcceb183 P 5182268877c348b7a48caffc5a6c695e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:16.300469 10002 raft_consensus.cc:729] T 5c55ffa5efdd4c78a44aa2addcceb183 P 5182268877c348b7a48caffc5a6c695e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5182268877c348b7a48caffc5a6c695e, State: Initialized, Role: FOLLOWER
I20260926 06:20:16.300580 10002 consensus_queue.cc:260] T 5c55ffa5efdd4c78a44aa2addcceb183 P 5182268877c348b7a48caffc5a6c695e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5182268877c348b7a48caffc5a6c695e" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 41233 } } peers { permanent_uuid: "72d08db2f2f440c6a153098d610864fb" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 42863 } } peers { permanent_uuid: "7380b48f51f948d58ba12ee3f695ab3e" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 38325 } }
I20260926 06:20:16.300848 10002 ts_tablet_manager.cc:1428] T 5c55ffa5efdd4c78a44aa2addcceb183 P 5182268877c348b7a48caffc5a6c695e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 06:20:16.300977 9855 heartbeater.cc:503] Master 127.4.202.190:36425 was elected leader, sending a full tablet report...
I20260926 06:20:16.300917 10001 raft_consensus.cc:348] T 5c55ffa5efdd4c78a44aa2addcceb183 P 72d08db2f2f440c6a153098d610864fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5182268877c348b7a48caffc5a6c695e" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 41233 } } peers { permanent_uuid: "72d08db2f2f440c6a153098d610864fb" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 42863 } } peers { permanent_uuid: "7380b48f51f948d58ba12ee3f695ab3e" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 38325 } }
I20260926 06:20:16.301062 10001 raft_consensus.cc:374] T 5c55ffa5efdd4c78a44aa2addcceb183 P 72d08db2f2f440c6a153098d610864fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:16.301095 10001 raft_consensus.cc:729] T 5c55ffa5efdd4c78a44aa2addcceb183 P 72d08db2f2f440c6a153098d610864fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72d08db2f2f440c6a153098d610864fb, State: Initialized, Role: FOLLOWER
I20260926 06:20:16.301226 10001 consensus_queue.cc:260] T 5c55ffa5efdd4c78a44aa2addcceb183 P 72d08db2f2f440c6a153098d610864fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5182268877c348b7a48caffc5a6c695e" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 41233 } } peers { permanent_uuid: "72d08db2f2f440c6a153098d610864fb" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 42863 } } peers { permanent_uuid: "7380b48f51f948d58ba12ee3f695ab3e" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 38325 } }
I20260926 06:20:16.300917 10000 raft_consensus.cc:348] T 5c55ffa5efdd4c78a44aa2addcceb183 P 7380b48f51f948d58ba12ee3f695ab3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5182268877c348b7a48caffc5a6c695e" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 41233 } } peers { permanent_uuid: "72d08db2f2f440c6a153098d610864fb" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 42863 } } peers { permanent_uuid: "7380b48f51f948d58ba12ee3f695ab3e" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 38325 } }
I20260926 06:20:16.301445 10000 raft_consensus.cc:374] T 5c55ffa5efdd4c78a44aa2addcceb183 P 7380b48f51f948d58ba12ee3f695ab3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:16.301496 10001 ts_tablet_manager.cc:1428] T 5c55ffa5efdd4c78a44aa2addcceb183 P 72d08db2f2f440c6a153098d610864fb: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 06:20:16.301489 10000 raft_consensus.cc:729] T 5c55ffa5efdd4c78a44aa2addcceb183 P 7380b48f51f948d58ba12ee3f695ab3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7380b48f51f948d58ba12ee3f695ab3e, State: Initialized, Role: FOLLOWER
I20260926 06:20:16.301604 10000 consensus_queue.cc:260] T 5c55ffa5efdd4c78a44aa2addcceb183 P 7380b48f51f948d58ba12ee3f695ab3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5182268877c348b7a48caffc5a6c695e" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 41233 } } peers { permanent_uuid: "72d08db2f2f440c6a153098d610864fb" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 42863 } } peers { permanent_uuid: "7380b48f51f948d58ba12ee3f695ab3e" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 38325 } }
I20260926 06:20:16.301755 9724 heartbeater.cc:503] Master 127.4.202.190:36425 was elected leader, sending a full tablet report...
I20260926 06:20:16.301860 9986 heartbeater.cc:503] Master 127.4.202.190:36425 was elected leader, sending a full tablet report...
I20260926 06:20:16.302100 10000 ts_tablet_manager.cc:1428] T 5c55ffa5efdd4c78a44aa2addcceb183 P 7380b48f51f948d58ba12ee3f695ab3e: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260926 06:20:16.370108 10008 raft_consensus.cc:482] T 5c55ffa5efdd4c78a44aa2addcceb183 P 7380b48f51f948d58ba12ee3f695ab3e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:20:16.370250 10008 raft_consensus.cc:504] T 5c55ffa5efdd4c78a44aa2addcceb183 P 7380b48f51f948d58ba12ee3f695ab3e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5182268877c348b7a48caffc5a6c695e" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 41233 } } peers { permanent_uuid: "72d08db2f2f440c6a153098d610864fb" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 42863 } } peers { permanent_uuid: "7380b48f51f948d58ba12ee3f695ab3e" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 38325 } }
I20260926 06:20:16.370647 10008 leader_election.cc:288] T 5c55ffa5efdd4c78a44aa2addcceb183 P 7380b48f51f948d58ba12ee3f695ab3e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5182268877c348b7a48caffc5a6c695e (127.4.202.130:41233), 72d08db2f2f440c6a153098d610864fb (127.4.202.129:42863)
I20260926 06:20:16.372022 9988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:16.370807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:42863 (local address 127.4.202.131:55287)
0926 06:20:16.370869 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:16.370880 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:16.370916 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:16.371095 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:16.371097 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:16.371099 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:16.371615 (+ 516us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:20:16.371616 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:16.371624 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:20:16.371638 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:16.371881 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:16.371882 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:16.371883 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 06:20:16.371937 (+ 54us) client_negotiation.cc:240] Negotiation successful
0926 06:20:16.371952 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260926 06:20:16.372061 9998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:16.370860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:55287 (local address 127.4.202.129:42863)
0926 06:20:16.370904 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:16.370906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:16.370916 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:20:16.371001 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:16.371002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:16.371009 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:16.371071 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:16.371748 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:16.371750 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:16.371750 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:20:16.371755 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:16.371775 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:20:16.371780 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:16.371851 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:16.371970 (+ 119us) server_negotiation.cc:299] Negotiation successful
0926 06:20:16.371986 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:20:16.372112 9997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:16.370881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:54085 (local address 127.4.202.130:41233)
0926 06:20:16.370921 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:16.370922 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:16.371078 (+ 156us) server_negotiation.cc:407] Connection header received
0926 06:20:16.371108 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:16.371109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:16.371114 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:16.371172 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:16.371886 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:16.371887 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:16.371887 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:20:16.371892 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:16.371908 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:20:16.371912 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:16.371966 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:16.372027 (+ 61us) server_negotiation.cc:299] Negotiation successful
0926 06:20:16.372039 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 06:20:16.372385 10009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:16.370808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:41233 (local address 127.4.202.131:54085)
0926 06:20:16.370991 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:16.371001 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:16.371060 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:16.371193 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:16.371194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:16.371195 (+ 1us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:16.371793 (+ 598us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:20:16.371794 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:16.371800 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:20:16.371812 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:16.371976 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:16.371977 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:16.371978 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 06:20:16.372028 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 06:20:16.372040 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":11,"thread_start_us":103,"threads_started":1}
I20260926 06:20:16.372380 9678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c55ffa5efdd4c78a44aa2addcceb183" candidate_uuid: "7380b48f51f948d58ba12ee3f695ab3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72d08db2f2f440c6a153098d610864fb" is_pre_election: true
I20260926 06:20:16.372530 9678 raft_consensus.cc:2454] T 5c55ffa5efdd4c78a44aa2addcceb183 P 72d08db2f2f440c6a153098d610864fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7380b48f51f948d58ba12ee3f695ab3e in term 0.
I20260926 06:20:16.372733 9809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c55ffa5efdd4c78a44aa2addcceb183" candidate_uuid: "7380b48f51f948d58ba12ee3f695ab3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5182268877c348b7a48caffc5a6c695e" is_pre_election: true
I20260926 06:20:16.372768 9873 leader_election.cc:302] T 5c55ffa5efdd4c78a44aa2addcceb183 P 7380b48f51f948d58ba12ee3f695ab3e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72d08db2f2f440c6a153098d610864fb, 7380b48f51f948d58ba12ee3f695ab3e; no voters:
I20260926 06:20:16.372875 9809 raft_consensus.cc:2454] T 5c55ffa5efdd4c78a44aa2addcceb183 P 5182268877c348b7a48caffc5a6c695e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7380b48f51f948d58ba12ee3f695ab3e in term 0.
I20260926 06:20:16.372898 10008 raft_consensus.cc:2781] T 5c55ffa5efdd4c78a44aa2addcceb183 P 7380b48f51f948d58ba12ee3f695ab3e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:20:16.372946 10008 raft_consensus.cc:482] T 5c55ffa5efdd4c78a44aa2addcceb183 P 7380b48f51f948d58ba12ee3f695ab3e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:20:16.372978 10008 raft_consensus.cc:3037] T 5c55ffa5efdd4c78a44aa2addcceb183 P 7380b48f51f948d58ba12ee3f695ab3e [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:16.373719 10008 raft_consensus.cc:504] T 5c55ffa5efdd4c78a44aa2addcceb183 P 7380b48f51f948d58ba12ee3f695ab3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5182268877c348b7a48caffc5a6c695e" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 41233 } } peers { permanent_uuid: "72d08db2f2f440c6a153098d610864fb" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 42863 } } peers { permanent_uuid: "7380b48f51f948d58ba12ee3f695ab3e" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 38325 } }
I20260926 06:20:16.373850 10008 leader_election.cc:288] T 5c55ffa5efdd4c78a44aa2addcceb183 P 7380b48f51f948d58ba12ee3f695ab3e [CANDIDATE]: Term 1 election: Requested vote from peers 5182268877c348b7a48caffc5a6c695e (127.4.202.130:41233), 72d08db2f2f440c6a153098d610864fb (127.4.202.129:42863)
I20260926 06:20:16.373986 9809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c55ffa5efdd4c78a44aa2addcceb183" candidate_uuid: "7380b48f51f948d58ba12ee3f695ab3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5182268877c348b7a48caffc5a6c695e"
I20260926 06:20:16.374001 9678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c55ffa5efdd4c78a44aa2addcceb183" candidate_uuid: "7380b48f51f948d58ba12ee3f695ab3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72d08db2f2f440c6a153098d610864fb"
I20260926 06:20:16.374086 9809 raft_consensus.cc:3037] T 5c55ffa5efdd4c78a44aa2addcceb183 P 5182268877c348b7a48caffc5a6c695e [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:16.374082 9678 raft_consensus.cc:3037] T 5c55ffa5efdd4c78a44aa2addcceb183 P 72d08db2f2f440c6a153098d610864fb [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:16.374842 9678 raft_consensus.cc:2454] T 5c55ffa5efdd4c78a44aa2addcceb183 P 72d08db2f2f440c6a153098d610864fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7380b48f51f948d58ba12ee3f695ab3e in term 1.
I20260926 06:20:16.374854 9809 raft_consensus.cc:2454] T 5c55ffa5efdd4c78a44aa2addcceb183 P 5182268877c348b7a48caffc5a6c695e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7380b48f51f948d58ba12ee3f695ab3e in term 1.
I20260926 06:20:16.374991 9873 leader_election.cc:302] T 5c55ffa5efdd4c78a44aa2addcceb183 P 7380b48f51f948d58ba12ee3f695ab3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72d08db2f2f440c6a153098d610864fb, 7380b48f51f948d58ba12ee3f695ab3e; no voters:
I20260926 06:20:16.375105 10008 raft_consensus.cc:2781] T 5c55ffa5efdd4c78a44aa2addcceb183 P 7380b48f51f948d58ba12ee3f695ab3e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:20:16.375229 10008 raft_consensus.cc:686] T 5c55ffa5efdd4c78a44aa2addcceb183 P 7380b48f51f948d58ba12ee3f695ab3e [term 1 LEADER]: Becoming Leader. State: Replica: 7380b48f51f948d58ba12ee3f695ab3e, State: Running, Role: LEADER
I20260926 06:20:16.375319 10008 consensus_queue.cc:237] T 5c55ffa5efdd4c78a44aa2addcceb183 P 7380b48f51f948d58ba12ee3f695ab3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5182268877c348b7a48caffc5a6c695e" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 41233 } } peers { permanent_uuid: "72d08db2f2f440c6a153098d610864fb" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 42863 } } peers { permanent_uuid: "7380b48f51f948d58ba12ee3f695ab3e" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 38325 } }
I20260926 06:20:16.376058 9535 catalog_manager.cc:5495] T 5c55ffa5efdd4c78a44aa2addcceb183 P 7380b48f51f948d58ba12ee3f695ab3e reported cstate change: term changed from 0 to 1, leader changed from <none> to 7380b48f51f948d58ba12ee3f695ab3e (127.4.202.131). New cstate: current_term: 1 leader_uuid: "7380b48f51f948d58ba12ee3f695ab3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5182268877c348b7a48caffc5a6c695e" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 41233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72d08db2f2f440c6a153098d610864fb" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 42863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7380b48f51f948d58ba12ee3f695ab3e" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 38325 } health_report { overall_health: HEALTHY } } }
I20260926 06:20:16.387141 9999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:16.386065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40854 (local address 127.4.202.131:38325)
0926 06:20:16.386114 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:16.386116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:16.386124 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:20:16.386206 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:16.386207 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:16.386217 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:16.386319 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:16.386935 (+ 616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:16.386936 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:16.386936 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:20:16.386941 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:16.386955 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:20:16.386959 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:16.387017 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:16.387064 (+ 47us) server_negotiation.cc:299] Negotiation successful
0926 06:20:16.387078 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 06:20:16.388617 9809 raft_consensus.cc:1261] T 5c55ffa5efdd4c78a44aa2addcceb183 P 5182268877c348b7a48caffc5a6c695e [term 1 FOLLOWER]: Refusing update from remote peer 7380b48f51f948d58ba12ee3f695ab3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:20:16.388619 9678 raft_consensus.cc:1261] T 5c55ffa5efdd4c78a44aa2addcceb183 P 72d08db2f2f440c6a153098d610864fb [term 1 FOLLOWER]: Refusing update from remote peer 7380b48f51f948d58ba12ee3f695ab3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:20:16.388949 10010 consensus_queue.cc:1035] T 5c55ffa5efdd4c78a44aa2addcceb183 P 7380b48f51f948d58ba12ee3f695ab3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "72d08db2f2f440c6a153098d610864fb" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 42863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:20:16.389091 10008 consensus_queue.cc:1035] T 5c55ffa5efdd4c78a44aa2addcceb183 P 7380b48f51f948d58ba12ee3f695ab3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5182268877c348b7a48caffc5a6c695e" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 41233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:20:16.392287 10014 mvcc.cc:205] Tried to move back new op lower bound from 7333493212725571584 to 7333493212673925120. Current Snapshot: MvccSnapshot[applied={T|T < 7333493212725571584}]
I20260926 06:20:16.393646 9536 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:20:16.393755 9536 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:20:16.394441 9536 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5ca1cffe650344dd83a3a33c537babe9: Sending DeleteTablet for 3 replicas of tablet 5c55ffa5efdd4c78a44aa2addcceb183
I20260926 06:20:16.394793 9789 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c55ffa5efdd4c78a44aa2addcceb183 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:20:16 UTC) from {username='slave'} at 127.0.0.1:51940
I20260926 06:20:16.394806 9920 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c55ffa5efdd4c78a44aa2addcceb183 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:20:16 UTC) from {username='slave'} at 127.0.0.1:40846
I20260926 06:20:16.395082 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 9596
I20260926 06:20:16.395071 9658 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c55ffa5efdd4c78a44aa2addcceb183 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:20:16 UTC) from {username='slave'} at 127.0.0.1:41134
I20260926 06:20:16.398706 10024 tablet_replica.cc:330] T 5c55ffa5efdd4c78a44aa2addcceb183 P 5182268877c348b7a48caffc5a6c695e: stopping tablet replica
I20260926 06:20:16.398813 10024 raft_consensus.cc:2229] T 5c55ffa5efdd4c78a44aa2addcceb183 P 5182268877c348b7a48caffc5a6c695e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:20:16.398913 10024 raft_consensus.cc:2258] T 5c55ffa5efdd4c78a44aa2addcceb183 P 5182268877c348b7a48caffc5a6c695e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:20:16.399060 10023 tablet_replica.cc:330] T 5c55ffa5efdd4c78a44aa2addcceb183 P 7380b48f51f948d58ba12ee3f695ab3e: stopping tablet replica
I20260926 06:20:16.399151 10023 raft_consensus.cc:2229] T 5c55ffa5efdd4c78a44aa2addcceb183 P 7380b48f51f948d58ba12ee3f695ab3e [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:20:16.399255 10023 raft_consensus.cc:2258] T 5c55ffa5efdd4c78a44aa2addcceb183 P 7380b48f51f948d58ba12ee3f695ab3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:20:16.399698 10024 ts_tablet_manager.cc:1905] T 5c55ffa5efdd4c78a44aa2addcceb183 P 5182268877c348b7a48caffc5a6c695e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:20:16.399698 10023 ts_tablet_manager.cc:1905] T 5c55ffa5efdd4c78a44aa2addcceb183 P 7380b48f51f948d58ba12ee3f695ab3e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 06:20:16.401031 9520 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.202.129:42863 (error 108)
I20260926 06:20:16.401557 10023 ts_tablet_manager.cc:1918] T 5c55ffa5efdd4c78a44aa2addcceb183 P 7380b48f51f948d58ba12ee3f695ab3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:20:16.401607 10024 ts_tablet_manager.cc:1918] T 5c55ffa5efdd4c78a44aa2addcceb183 P 5182268877c348b7a48caffc5a6c695e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:20:16.401726 10023 log.cc:1196] T 5c55ffa5efdd4c78a44aa2addcceb183 P 7380b48f51f948d58ba12ee3f695ab3e: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790403591640726-4906-0/minicluster-data/ts-2/wal/wals/5c55ffa5efdd4c78a44aa2addcceb183
I20260926 06:20:16.401746 9996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:16.401630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:42863 (local address 127.0.0.1:41140)
0926 06:20:16.401675 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:16.401716 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.129:42863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260926 06:20:16.401748 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 9727
W20260926 06:20:16.402340 9520 catalog_manager.cc:4584] TS 72d08db2f2f440c6a153098d610864fb (127.4.202.129:42863): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5c55ffa5efdd4c78a44aa2addcceb183: Network error: Client connection negotiation failed: client connection to 127.4.202.129:42863: connect: Connection refused (error 111)
I20260926 06:20:16.402549 10023 ts_tablet_manager.cc:1939] T 5c55ffa5efdd4c78a44aa2addcceb183 P 7380b48f51f948d58ba12ee3f695ab3e: Deleting consensus metadata
I20260926 06:20:16.402966 9522 catalog_manager.cc:4841] TS 7380b48f51f948d58ba12ee3f695ab3e (127.4.202.131:38325): tablet 5c55ffa5efdd4c78a44aa2addcceb183 (table test-table [id=53ab13e314e740f49e783d4ceadaad81]) successfully deleted
I20260926 06:20:16.405680 9996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:16.405561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:41233 (local address 127.0.0.1:51954)
0926 06:20:16.405623 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:16.405651 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.130:41233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260926 06:20:16.406388 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 9858
I20260926 06:20:16.411307 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 9505
2026-09-26T06:20:16Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (638 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260926 06:20:16.431113 4906 mini_oidc.cc:152] Starting JWKS server
I20260926 06:20:16.432126 4906 webserver.cc:460] Webserver started at https://127.0.0.1:44543/ using document root <none> and password file <none>
I20260926 06:20:16.432273 4906 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 06:20:16.432453 4906 webserver.cc:460] Webserver started at http://127.0.0.1:42013/ using document root <none> and password file <none>
2026-09-26T06:20:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:20:16Z Disabled control of system clock
I20260926 06:20:16.439779 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:36521
--webserver_interface=127.4.202.190
--webserver_port=0
--builtin_ntp_servers=127.4.202.148:46661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:36521
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44543/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790403591640726-4906-0/testchainca.cert with env {}
W20260926 06:20:16.521395 10031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:16.521576 10031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:16.521597 10031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:16.522814 10031 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:20:16.522854 10031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:16.522866 10031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:16.522879 10031 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:20:16.522924 10031 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 06:20:16.522940 10031 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 06:20:16.522965 10031 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44543/jwks.json
I20260926 06:20:16.524547 10031 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:46661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:36521
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:36521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44543/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790403591640726-4906-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:16.524786 10031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:16.524998 10031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:16.529820 10037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:16.529845 10036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:16.529865 10031 server_base.cc:1034] running on GCE node
W20260926 06:20:16.529847 10039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:16.530217 10031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:16.530491 10031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:16.531634 10031 hybrid_clock.cc:648] HybridClock initialized: now 1790403616531618 us; error 32 us; skew 500 ppm
I20260926 06:20:16.532790 10031 webserver.cc:460] Webserver started at http://127.4.202.190:41089/ using document root <none> and password file <none>
I20260926 06:20:16.533035 10031 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:16.533098 10031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:16.533250 10031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:16.534250 10031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "17369d618b2144ad9c632456d88b877e"
format_stamp: "Formatted at 2026-09-26 06:20:16 on dist-test-slave-pd0t"
server_key: "7996cc06e2eb84d7ab72e3ae1f72caa6"
server_key_iv: "b085714b7e82b1c2beb9b6cc38b90d8b"
server_key_version: "encryptionkey@0"
I20260926 06:20:16.534641 10031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "17369d618b2144ad9c632456d88b877e"
format_stamp: "Formatted at 2026-09-26 06:20:16 on dist-test-slave-pd0t"
server_key: "7996cc06e2eb84d7ab72e3ae1f72caa6"
server_key_iv: "b085714b7e82b1c2beb9b6cc38b90d8b"
server_key_version: "encryptionkey@0"
I20260926 06:20:16.535924 10031 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:20:16.536629 10045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:16.536798 10031 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:20:16.536893 10031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "17369d618b2144ad9c632456d88b877e"
format_stamp: "Formatted at 2026-09-26 06:20:16 on dist-test-slave-pd0t"
server_key: "7996cc06e2eb84d7ab72e3ae1f72caa6"
server_key_iv: "b085714b7e82b1c2beb9b6cc38b90d8b"
server_key_version: "encryptionkey@0"
I20260926 06:20:16.536978 10031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:16.548008 10031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:16.548270 10031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:16.548411 10031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:16.552811 10031 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:36521
I20260926 06:20:16.553202 10031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:20:16.553697 10097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:36521 every 8 connection(s)
I20260926 06:20:16.554522 10098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:16.557219 10098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17369d618b2144ad9c632456d88b877e: Bootstrap starting.
I20260926 06:20:16.557827 10098 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17369d618b2144ad9c632456d88b877e: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:16.558101 10098 log.cc:824] T 00000000000000000000000000000000 P 17369d618b2144ad9c632456d88b877e: Log is configured to *not* fsync() on all Append() calls
I20260926 06:20:16.558900 10098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17369d618b2144ad9c632456d88b877e: No bootstrap required, opened a new log
I20260926 06:20:16.559078 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 10031
I20260926 06:20:16.559160 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:20:16.559329 4906 external_mini_cluster.cc:1428] Setting key 53bce62cc8c1aefd8158c9843558e08c
I20260926 06:20:16.561122 10098 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17369d618b2144ad9c632456d88b877e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17369d618b2144ad9c632456d88b877e" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 36521 } }
I20260926 06:20:16.561286 10098 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17369d618b2144ad9c632456d88b877e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:16.561326 10098 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17369d618b2144ad9c632456d88b877e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17369d618b2144ad9c632456d88b877e, State: Initialized, Role: FOLLOWER
I20260926 06:20:16.561439 10098 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17369d618b2144ad9c632456d88b877e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17369d618b2144ad9c632456d88b877e" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 36521 } }
I20260926 06:20:16.561514 10098 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17369d618b2144ad9c632456d88b877e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:20:16.561544 10098 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17369d618b2144ad9c632456d88b877e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:20:16.561582 10098 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17369d618b2144ad9c632456d88b877e [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:16.562139 10098 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17369d618b2144ad9c632456d88b877e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17369d618b2144ad9c632456d88b877e" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 36521 } }
I20260926 06:20:16.562253 10098 leader_election.cc:302] T 00000000000000000000000000000000 P 17369d618b2144ad9c632456d88b877e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17369d618b2144ad9c632456d88b877e; no voters:
I20260926 06:20:16.562409 10098 leader_election.cc:288] T 00000000000000000000000000000000 P 17369d618b2144ad9c632456d88b877e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:20:16.562634 10103 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17369d618b2144ad9c632456d88b877e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:20:16.562676 10098 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17369d618b2144ad9c632456d88b877e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:20:16.562758 10103 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17369d618b2144ad9c632456d88b877e [term 1 LEADER]: Becoming Leader. State: Replica: 17369d618b2144ad9c632456d88b877e, State: Running, Role: LEADER
I20260926 06:20:16.562870 10103 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17369d618b2144ad9c632456d88b877e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17369d618b2144ad9c632456d88b877e" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 36521 } }
I20260926 06:20:16.563194 10104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17369d618b2144ad9c632456d88b877e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17369d618b2144ad9c632456d88b877e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17369d618b2144ad9c632456d88b877e" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 36521 } } }
I20260926 06:20:16.563290 10104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17369d618b2144ad9c632456d88b877e [sys.catalog]: This master's current role is: LEADER
I20260926 06:20:16.563182 10103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17369d618b2144ad9c632456d88b877e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17369d618b2144ad9c632456d88b877e. Latest consensus state: current_term: 1 leader_uuid: "17369d618b2144ad9c632456d88b877e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17369d618b2144ad9c632456d88b877e" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 36521 } } }
I20260926 06:20:16.563458 10103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17369d618b2144ad9c632456d88b877e [sys.catalog]: This master's current role is: LEADER
I20260926 06:20:16.563673 10112 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:20:16.564146 10112 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:20:16.565522 10120 mvcc.cc:205] Tried to move back new op lower bound from 7333493213450719232 to 7333493213441871872. Current Snapshot: MvccSnapshot[applied={T|T < 7333493213450719232}]
I20260926 06:20:16.565907 10112 catalog_manager.cc:1347] Generated new cluster ID: 0c634be5298e4c6ea5cf4f3b1437b769
I20260926 06:20:16.565958 10112 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:20:16.567456 10102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:16.560888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53876 (local address 127.4.202.190:36521)
0926 06:20:16.561798 (+ 910us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:16.561807 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:16.561845 (+ 38us) server_negotiation.cc:407] Connection header received
0926 06:20:16.561930 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:16.561935 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:16.561992 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:16.562093 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:16.562381 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:16.563866 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:16.565032 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:16.565650 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:16.565751 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:16.566723 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:16.566726 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:16.566727 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:20:16.566904 (+ 177us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:16.566939 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:20:16.566951 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:16.566984 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:16.567124 (+ 140us) server_negotiation.cc:299] Negotiation successful
0926 06:20:16.567223 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":813,"thread_start_us":53,"threads_started":1}
I20260926 06:20:16.571473 4906 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:20:16.586908 10112 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:20:16.587378 10112 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:20:16.593395 10112 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17369d618b2144ad9c632456d88b877e: Generated new TSK 0
I20260926 06:20:16.593564 10112 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260926 06:20:16.616878 10128 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260926 06:20:16.616967 10128 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 06:20:16.617177 10102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:16.601244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53884 (local address 127.4.202.190:36521)
0926 06:20:16.601300 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:16.601302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:16.601467 (+ 165us) server_negotiation.cc:407] Connection header received
0926 06:20:16.601533 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:16.601534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:16.601546 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:16.601621 (+ 75us) server_negotiation.cc:226] Negotiated authn=JWT
0926 06:20:16.601810 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:16.602614 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:16.603355 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:16.603670 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:16.603738 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:16.603921 (+ 183us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 06:20:16.616994 (+ 13073us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44543/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0926 06:20:16.617115 (+ 121us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53884: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44543/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":33}
W20260926 06:20:16.617230 10102 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53884: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44543/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260926 06:20:16.617606 10127 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:20:16.601158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:36521 (local address 127.0.0.1:53884)
0926 06:20:16.601407 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:16.601416 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:16.601468 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:16.601663 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:16.601664 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:16.601666 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0926 06:20:16.601721 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:16.601723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:16.602691 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:16.602691 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:16.603277 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:16.603279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:16.603755 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:16.603756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:16.603804 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:16.603821 (+ 17us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 06:20:16.617114 (+ 13293us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44543/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0926 06:20:16.617202 (+ 88us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.202.190:36521: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44543/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":173,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.4.202.190:36521: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44543/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260926 06:20:16.618114 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 10031
2026-09-26T06:20:16Z chronyd exiting
I20260926 06:20:16.653432 4906 test_util.cc:175] -----------------------------------------------
I20260926 06:20:16.653486 4906 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790403591640726-4906-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (223 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260926 06:20:16.654120 4906 mini_oidc.cc:152] Starting JWKS server
I20260926 06:20:16.654807 4906 webserver.cc:460] Webserver started at https://127.0.0.1:37485/ using document root <none> and password file <none>
I20260926 06:20:16.654846 4906 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 06:20:16.654953 4906 webserver.cc:460] Webserver started at http://127.0.0.1:42713/ using document root <none> and password file <none>
2026-09-26T06:20:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:20:16Z Disabled control of system clock
I20260926 06:20:16.662067 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:45549
--webserver_interface=127.4.202.190
--webserver_port=0
--builtin_ntp_servers=127.4.202.148:32999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:45549
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37485/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790403591640726-4906-0/testchainca.cert with env {}
W20260926 06:20:16.739817 10134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:16.740000 10134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:16.740020 10134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:16.741091 10134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:20:16.741122 10134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:16.741134 10134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:16.741146 10134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:20:16.741158 10134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 06:20:16.741169 10134 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 06:20:16.741181 10134 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37485/jwks.json
I20260926 06:20:16.742515 10134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:32999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:45549
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:45549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37485/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790403591640726-4906-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:16.742710 10134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:16.742862 10134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:16.747381 10140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:16.747370 10142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:16.747447 10134 server_base.cc:1034] running on GCE node
W20260926 06:20:16.747381 10139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:16.747807 10134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:16.748064 10134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:16.749194 10134 hybrid_clock.cc:648] HybridClock initialized: now 1790403616749163 us; error 48 us; skew 500 ppm
I20260926 06:20:16.750355 10134 webserver.cc:460] Webserver started at http://127.4.202.190:35183/ using document root <none> and password file <none>
I20260926 06:20:16.750571 10134 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:16.750689 10134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:16.750828 10134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:16.751791 10134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "63061e41d22e49e0850a22b04702e1d6"
format_stamp: "Formatted at 2026-09-26 06:20:16 on dist-test-slave-pd0t"
server_key: "4c53910727be4df2ec26b1a23071ea17"
server_key_iv: "28c9264773e919ad9bbccdde9d715ef4"
server_key_version: "encryptionkey@0"
I20260926 06:20:16.752132 10134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "63061e41d22e49e0850a22b04702e1d6"
format_stamp: "Formatted at 2026-09-26 06:20:16 on dist-test-slave-pd0t"
server_key: "4c53910727be4df2ec26b1a23071ea17"
server_key_iv: "28c9264773e919ad9bbccdde9d715ef4"
server_key_version: "encryptionkey@0"
I20260926 06:20:16.753396 10134 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:20:16.754057 10148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:16.754227 10134 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:20:16.754321 10134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "63061e41d22e49e0850a22b04702e1d6"
format_stamp: "Formatted at 2026-09-26 06:20:16 on dist-test-slave-pd0t"
server_key: "4c53910727be4df2ec26b1a23071ea17"
server_key_iv: "28c9264773e919ad9bbccdde9d715ef4"
server_key_version: "encryptionkey@0"
I20260926 06:20:16.754395 10134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:16.763092 10134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:16.763304 10134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:16.763396 10134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:16.767055 10134 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:45549
I20260926 06:20:16.767082 10200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:45549 every 8 connection(s)
I20260926 06:20:16.767374 10134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:20:16.768190 10201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:16.770251 10201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63061e41d22e49e0850a22b04702e1d6: Bootstrap starting.
I20260926 06:20:16.770859 10201 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63061e41d22e49e0850a22b04702e1d6: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:16.771136 10201 log.cc:824] T 00000000000000000000000000000000 P 63061e41d22e49e0850a22b04702e1d6: Log is configured to *not* fsync() on all Append() calls
I20260926 06:20:16.771919 10201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63061e41d22e49e0850a22b04702e1d6: No bootstrap required, opened a new log
I20260926 06:20:16.773463 10201 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63061e41d22e49e0850a22b04702e1d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63061e41d22e49e0850a22b04702e1d6" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45549 } }
I20260926 06:20:16.773589 10201 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63061e41d22e49e0850a22b04702e1d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:16.773610 10201 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63061e41d22e49e0850a22b04702e1d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63061e41d22e49e0850a22b04702e1d6, State: Initialized, Role: FOLLOWER
I20260926 06:20:16.773778 10201 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63061e41d22e49e0850a22b04702e1d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63061e41d22e49e0850a22b04702e1d6" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45549 } }
I20260926 06:20:16.773837 10201 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63061e41d22e49e0850a22b04702e1d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:20:16.773859 10201 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63061e41d22e49e0850a22b04702e1d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:20:16.773928 10201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63061e41d22e49e0850a22b04702e1d6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:16.774417 10201 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63061e41d22e49e0850a22b04702e1d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63061e41d22e49e0850a22b04702e1d6" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45549 } }
I20260926 06:20:16.774524 10201 leader_election.cc:302] T 00000000000000000000000000000000 P 63061e41d22e49e0850a22b04702e1d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63061e41d22e49e0850a22b04702e1d6; no voters:
I20260926 06:20:16.774724 10201 leader_election.cc:288] T 00000000000000000000000000000000 P 63061e41d22e49e0850a22b04702e1d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:20:16.774768 10204 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63061e41d22e49e0850a22b04702e1d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:20:16.774937 10204 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63061e41d22e49e0850a22b04702e1d6 [term 1 LEADER]: Becoming Leader. State: Replica: 63061e41d22e49e0850a22b04702e1d6, State: Running, Role: LEADER
I20260926 06:20:16.774959 10201 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63061e41d22e49e0850a22b04702e1d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:20:16.775027 10204 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63061e41d22e49e0850a22b04702e1d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63061e41d22e49e0850a22b04702e1d6" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45549 } }
I20260926 06:20:16.775426 10205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63061e41d22e49e0850a22b04702e1d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63061e41d22e49e0850a22b04702e1d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63061e41d22e49e0850a22b04702e1d6" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45549 } } }
I20260926 06:20:16.775530 10205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63061e41d22e49e0850a22b04702e1d6 [sys.catalog]: This master's current role is: LEADER
I20260926 06:20:16.775528 10206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63061e41d22e49e0850a22b04702e1d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63061e41d22e49e0850a22b04702e1d6. Latest consensus state: current_term: 1 leader_uuid: "63061e41d22e49e0850a22b04702e1d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63061e41d22e49e0850a22b04702e1d6" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45549 } } }
I20260926 06:20:16.775771 10206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63061e41d22e49e0850a22b04702e1d6 [sys.catalog]: This master's current role is: LEADER
I20260926 06:20:16.775772 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 10134
I20260926 06:20:16.775858 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:20:16.776099 4906 external_mini_cluster.cc:1428] Setting key 6679bb2d0d9467d8c60c9b881a5bc03d
I20260926 06:20:16.776197 10213 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:20:16.776713 10213 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:20:16.778079 10213 catalog_manager.cc:1347] Generated new cluster ID: 058db36bd04f4c458d0515e235a0cca0
I20260926 06:20:16.778133 10213 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:20:16.781088 10223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:16.776536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40334 (local address 127.4.202.190:45549)
0926 06:20:16.776892 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:16.776899 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:16.776921 (+ 22us) server_negotiation.cc:407] Connection header received
0926 06:20:16.777003 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:16.777007 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:16.777054 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:16.777187 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:16.777349 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:16.778464 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:16.779293 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:16.779699 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:16.779829 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:16.780489 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:16.780493 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:16.780493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:20:16.780666 (+ 173us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:16.780695 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:20:16.780704 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:16.780779 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:16.780848 (+ 69us) server_negotiation.cc:299] Negotiation successful
0926 06:20:16.780919 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":67,"threads_started":1}
I20260926 06:20:16.781870 4906 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:20:16.800400 10213 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:20:16.800886 10213 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:20:16.804831 10213 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63061e41d22e49e0850a22b04702e1d6: Generated new TSK 0
I20260926 06:20:16.804986 10213 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260926 06:20:16.827943 10231 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260926 06:20:16.828013 10231 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 06:20:16.828290 10223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:16.814321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40346 (local address 127.4.202.190:45549)
0926 06:20:16.814364 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:16.814366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:16.814469 (+ 103us) server_negotiation.cc:407] Connection header received
0926 06:20:16.814507 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:16.814508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:16.814520 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:16.814622 (+ 102us) server_negotiation.cc:226] Negotiated authn=JWT
0926 06:20:16.814774 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:16.815581 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:16.816307 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:16.816613 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:16.816683 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:16.816839 (+ 156us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 06:20:16.828101 (+ 11262us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37485/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0926 06:20:16.828229 (+ 128us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40346: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37485/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":24}
W20260926 06:20:16.828342 10223 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40346: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37485/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260926 06:20:16.828357 10230 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:20:16.814240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:45549 (local address 127.0.0.1:40346)
0926 06:20:16.814389 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:16.814399 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:16.814463 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:16.814637 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:16.814638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:16.814640 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0926 06:20:16.814697 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:16.814699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:16.815657 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:16.815657 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:16.816217 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:16.816218 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:16.816688 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:16.816690 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:16.816738 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:16.816742 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 06:20:16.828194 (+ 11452us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37485/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0926 06:20:16.828273 (+ 79us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.202.190:45549: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37485/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":103,"threads_started":1}
I20260926 06:20:16.828581 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 10134
2026-09-26T06:20:16Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (211 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260926 06:20:16.865545 4906 mini_oidc.cc:152] Starting JWKS server
I20260926 06:20:16.866214 4906 webserver.cc:460] Webserver started at https://127.0.0.1:42877/ using document root <none> and password file <none>
I20260926 06:20:16.866310 4906 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 06:20:16.866505 4906 webserver.cc:460] Webserver started at http://127.0.0.1:45615/ using document root <none> and password file <none>
2026-09-26T06:20:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:20:16Z Disabled control of system clock
I20260926 06:20:16.873857 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:45653
--webserver_interface=127.4.202.190
--webserver_port=0
--builtin_ntp_servers=127.4.202.148:36397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:45653
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42877/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260926 06:20:16.950198 10237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:16.950389 10237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:16.950409 10237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:16.951457 10237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:20:16.951491 10237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:16.951503 10237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:16.951515 10237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:20:16.951527 10237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 06:20:16.951539 10237 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 06:20:16.951560 10237 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42877/jwks.json
I20260926 06:20:16.952850 10237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:36397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:45653
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:45653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42877/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:16.953027 10237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:16.953248 10237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:16.957772 10242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:16.957867 10237 server_base.cc:1034] running on GCE node
W20260926 06:20:16.957762 10245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:16.957850 10243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:16.958223 10237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:16.958454 10237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:16.959594 10237 hybrid_clock.cc:648] HybridClock initialized: now 1790403616959573 us; error 41 us; skew 500 ppm
I20260926 06:20:16.960716 10237 webserver.cc:460] Webserver started at http://127.4.202.190:38939/ using document root <none> and password file <none>
I20260926 06:20:16.960940 10237 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:16.961006 10237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:16.961148 10237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:16.962116 10237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "2c94c05590244fafa2f3ea5aa62695ff"
format_stamp: "Formatted at 2026-09-26 06:20:16 on dist-test-slave-pd0t"
server_key: "25197fd3a5582820dcfc872f2757aa91"
server_key_iv: "1b464092d1ab42a71bc3ebbc15f1bfc5"
server_key_version: "encryptionkey@0"
I20260926 06:20:16.962461 10237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "2c94c05590244fafa2f3ea5aa62695ff"
format_stamp: "Formatted at 2026-09-26 06:20:16 on dist-test-slave-pd0t"
server_key: "25197fd3a5582820dcfc872f2757aa91"
server_key_iv: "1b464092d1ab42a71bc3ebbc15f1bfc5"
server_key_version: "encryptionkey@0"
I20260926 06:20:16.963702 10237 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:20:16.964380 10251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:16.964555 10237 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:20:16.964648 10237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "2c94c05590244fafa2f3ea5aa62695ff"
format_stamp: "Formatted at 2026-09-26 06:20:16 on dist-test-slave-pd0t"
server_key: "25197fd3a5582820dcfc872f2757aa91"
server_key_iv: "1b464092d1ab42a71bc3ebbc15f1bfc5"
server_key_version: "encryptionkey@0"
I20260926 06:20:16.964731 10237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:16.976621 10237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:16.976879 10237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:16.977020 10237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:16.980785 10237 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:45653
I20260926 06:20:16.980819 10303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:45653 every 8 connection(s)
I20260926 06:20:16.981130 10237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:20:16.981758 10304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:16.984414 10304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c94c05590244fafa2f3ea5aa62695ff: Bootstrap starting.
I20260926 06:20:16.984961 10304 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c94c05590244fafa2f3ea5aa62695ff: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:16.985250 10304 log.cc:824] T 00000000000000000000000000000000 P 2c94c05590244fafa2f3ea5aa62695ff: Log is configured to *not* fsync() on all Append() calls
I20260926 06:20:16.985893 10304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c94c05590244fafa2f3ea5aa62695ff: No bootstrap required, opened a new log
I20260926 06:20:16.987430 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 10237
I20260926 06:20:16.987545 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:20:16.987510 10304 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c94c05590244fafa2f3ea5aa62695ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c94c05590244fafa2f3ea5aa62695ff" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45653 } }
I20260926 06:20:16.987718 10304 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c94c05590244fafa2f3ea5aa62695ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:16.987761 4906 external_mini_cluster.cc:1428] Setting key 0f3355f98f72020af6d6ad050d7d80bb
I20260926 06:20:16.987780 10304 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c94c05590244fafa2f3ea5aa62695ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c94c05590244fafa2f3ea5aa62695ff, State: Initialized, Role: FOLLOWER
I20260926 06:20:16.987928 10304 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c94c05590244fafa2f3ea5aa62695ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c94c05590244fafa2f3ea5aa62695ff" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45653 } }
I20260926 06:20:16.988018 10304 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c94c05590244fafa2f3ea5aa62695ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:20:16.988065 10304 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c94c05590244fafa2f3ea5aa62695ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:20:16.988119 10304 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c94c05590244fafa2f3ea5aa62695ff [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:16.988775 10304 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c94c05590244fafa2f3ea5aa62695ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c94c05590244fafa2f3ea5aa62695ff" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45653 } }
I20260926 06:20:16.988926 10304 leader_election.cc:302] T 00000000000000000000000000000000 P 2c94c05590244fafa2f3ea5aa62695ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c94c05590244fafa2f3ea5aa62695ff; no voters:
I20260926 06:20:16.989078 10304 leader_election.cc:288] T 00000000000000000000000000000000 P 2c94c05590244fafa2f3ea5aa62695ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:20:16.989107 10309 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c94c05590244fafa2f3ea5aa62695ff [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:20:16.989259 10309 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c94c05590244fafa2f3ea5aa62695ff [term 1 LEADER]: Becoming Leader. State: Replica: 2c94c05590244fafa2f3ea5aa62695ff, State: Running, Role: LEADER
I20260926 06:20:16.989394 10304 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c94c05590244fafa2f3ea5aa62695ff [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:20:16.989365 10309 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c94c05590244fafa2f3ea5aa62695ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c94c05590244fafa2f3ea5aa62695ff" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45653 } }
I20260926 06:20:16.989848 10311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c94c05590244fafa2f3ea5aa62695ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c94c05590244fafa2f3ea5aa62695ff. Latest consensus state: current_term: 1 leader_uuid: "2c94c05590244fafa2f3ea5aa62695ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c94c05590244fafa2f3ea5aa62695ff" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45653 } } }
I20260926 06:20:16.990080 10311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c94c05590244fafa2f3ea5aa62695ff [sys.catalog]: This master's current role is: LEADER
I20260926 06:20:16.990007 10310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c94c05590244fafa2f3ea5aa62695ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c94c05590244fafa2f3ea5aa62695ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c94c05590244fafa2f3ea5aa62695ff" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45653 } } }
I20260926 06:20:16.990329 10310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c94c05590244fafa2f3ea5aa62695ff [sys.catalog]: This master's current role is: LEADER
I20260926 06:20:16.990483 10318 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:20:16.991070 10318 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:20:16.992602 10318 catalog_manager.cc:1347] Generated new cluster ID: 515268274b3b47fdb3783c71fe7de5a8
I20260926 06:20:16.992657 10318 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:20:16.994241 10308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:16.988354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36822 (local address 127.4.202.190:45653)
0926 06:20:16.988536 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:16.988543 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:16.988565 (+ 22us) server_negotiation.cc:407] Connection header received
0926 06:20:16.988649 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:16.988654 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:16.988706 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:16.988816 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:16.989099 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:16.990968 (+ 1869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:16.991945 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:16.992714 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:16.992839 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:16.993533 (+ 694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:16.993537 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:16.993537 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:20:16.993732 (+ 195us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:16.993770 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:20:16.993783 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:16.993846 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:16.993976 (+ 130us) server_negotiation.cc:299] Negotiation successful
0926 06:20:16.994046 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
I20260926 06:20:16.995182 4906 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:20:17.006693 10318 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:20:17.007156 10318 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:20:17.014598 10318 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c94c05590244fafa2f3ea5aa62695ff: Generated new TSK 0
I20260926 06:20:17.014766 10318 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:20:17.027211 10308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:17.023608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36830 (local address 127.4.202.190:45653)
0926 06:20:17.023668 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:17.023671 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:17.023795 (+ 124us) server_negotiation.cc:407] Connection header received
0926 06:20:17.023862 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:17.023863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:17.023873 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:17.023935 (+ 62us) server_negotiation.cc:226] Negotiated authn=JWT
0926 06:20:17.024078 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:17.024540 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:17.025639 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:17.025829 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:17.025877 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:17.026060 (+ 183us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 06:20:17.027001 (+ 941us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42877/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0926 06:20:17.027140 (+ 139us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36830: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42877/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":38}
W20260926 06:20:17.027276 10308 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36830: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42877/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260926 06:20:17.027307 10333 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:20:17.023556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:45653 (local address 127.0.0.1:36830)
0926 06:20:17.023721 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:17.023733 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:17.023798 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:17.023948 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:17.023949 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:17.023952 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0926 06:20:17.024016 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:17.024018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:17.024621 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:17.024622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:17.025560 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:17.025563 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:17.025903 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:17.025904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:17.025971 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:17.025976 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 06:20:17.027108 (+ 1132us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42877/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0926 06:20:17.027190 (+ 82us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.202.190:45653: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42877/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":96,"threads_started":1}
I20260926 06:20:17.027508 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 10237
E20260926 06:20:17.037642 10334 webserver.cc:533] Webserver: sslize failed:
E20260926 06:20:17.037694 10334 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-26T06:20:17Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (199 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-26T06:20:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:20:17Z Disabled control of system clock
I20260926 06:20:17.072446 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:46863
--webserver_interface=127.4.202.190
--webserver_port=0
--builtin_ntp_servers=127.4.202.148:36881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:46863
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790403591640726-4906-0/testchainca.cert with env {}
W20260926 06:20:17.153143 10338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:17.153327 10338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:17.153348 10338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:17.154942 10338 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:20:17.154994 10338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:17.155038 10338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:17.155064 10338 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:20:17.155092 10338 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 06:20:17.155122 10338 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 06:20:17.155150 10338 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260926 06:20:17.156580 10338 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:36881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:46863
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:46863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790403591640726-4906-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:17.156808 10338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:17.157024 10338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:17.161641 10344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:17.161711 10343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:17.161731 10346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:17.161922 10338 server_base.cc:1034] running on GCE node
I20260926 06:20:17.162169 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:17.162456 10338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:17.163621 10338 hybrid_clock.cc:648] HybridClock initialized: now 1790403617163607 us; error 42 us; skew 500 ppm
I20260926 06:20:17.164776 10338 webserver.cc:460] Webserver started at http://127.4.202.190:39759/ using document root <none> and password file <none>
I20260926 06:20:17.165023 10338 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:17.165091 10338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:17.165249 10338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:17.166370 10338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "f04cc9de19e842fdbfe8b42a03561182"
format_stamp: "Formatted at 2026-09-26 06:20:17 on dist-test-slave-pd0t"
server_key: "6fa683f5005b1cab89716bd7a8a06e28"
server_key_iv: "9062516ceb0a67d8623e359134520cc1"
server_key_version: "encryptionkey@0"
I20260926 06:20:17.166852 10338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "f04cc9de19e842fdbfe8b42a03561182"
format_stamp: "Formatted at 2026-09-26 06:20:17 on dist-test-slave-pd0t"
server_key: "6fa683f5005b1cab89716bd7a8a06e28"
server_key_iv: "9062516ceb0a67d8623e359134520cc1"
server_key_version: "encryptionkey@0"
I20260926 06:20:17.168152 10338 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:20:17.168856 10352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:17.169045 10338 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:20:17.169145 10338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "f04cc9de19e842fdbfe8b42a03561182"
format_stamp: "Formatted at 2026-09-26 06:20:17 on dist-test-slave-pd0t"
server_key: "6fa683f5005b1cab89716bd7a8a06e28"
server_key_iv: "9062516ceb0a67d8623e359134520cc1"
server_key_version: "encryptionkey@0"
I20260926 06:20:17.169224 10338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:17.189977 10338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:17.190286 10338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:17.190436 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:17.194135 10338 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:46863
I20260926 06:20:17.194208 10404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:46863 every 8 connection(s)
I20260926 06:20:17.194528 10338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:20:17.195493 10405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:17.196409 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 10338
I20260926 06:20:17.196517 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:20:17.196695 4906 external_mini_cluster.cc:1428] Setting key 458ca9df2a713681a35b41fd828a4402
I20260926 06:20:17.198223 10405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f04cc9de19e842fdbfe8b42a03561182: Bootstrap starting.
I20260926 06:20:17.198886 10405 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f04cc9de19e842fdbfe8b42a03561182: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:17.199183 10405 log.cc:824] T 00000000000000000000000000000000 P f04cc9de19e842fdbfe8b42a03561182: Log is configured to *not* fsync() on all Append() calls
I20260926 06:20:17.199847 10405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f04cc9de19e842fdbfe8b42a03561182: No bootstrap required, opened a new log
I20260926 06:20:17.201525 10405 raft_consensus.cc:348] T 00000000000000000000000000000000 P f04cc9de19e842fdbfe8b42a03561182 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f04cc9de19e842fdbfe8b42a03561182" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 46863 } }
I20260926 06:20:17.201676 10405 raft_consensus.cc:374] T 00000000000000000000000000000000 P f04cc9de19e842fdbfe8b42a03561182 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:17.201730 10405 raft_consensus.cc:729] T 00000000000000000000000000000000 P f04cc9de19e842fdbfe8b42a03561182 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f04cc9de19e842fdbfe8b42a03561182, State: Initialized, Role: FOLLOWER
I20260926 06:20:17.201891 10405 consensus_queue.cc:260] T 00000000000000000000000000000000 P f04cc9de19e842fdbfe8b42a03561182 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f04cc9de19e842fdbfe8b42a03561182" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 46863 } }
I20260926 06:20:17.202003 10405 raft_consensus.cc:388] T 00000000000000000000000000000000 P f04cc9de19e842fdbfe8b42a03561182 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:20:17.202049 10405 raft_consensus.cc:482] T 00000000000000000000000000000000 P f04cc9de19e842fdbfe8b42a03561182 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:20:17.202061 10408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:17.197192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35928 (local address 127.4.202.190:46863)
0926 06:20:17.197401 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:17.197410 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:17.197442 (+ 32us) server_negotiation.cc:407] Connection header received
0926 06:20:17.197937 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:17.197945 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:17.198022 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:17.198174 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:17.198362 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:17.199442 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:17.200236 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:17.200616 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:17.200761 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:17.201421 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:17.201425 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:17.201425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:20:17.201592 (+ 167us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:17.201621 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:20:17.201631 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:17.201707 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:17.201808 (+ 101us) server_negotiation.cc:299] Negotiation successful
0926 06:20:17.201892 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":78,"threads_started":1}
I20260926 06:20:17.202101 10405 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f04cc9de19e842fdbfe8b42a03561182 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:17.202816 10405 raft_consensus.cc:504] T 00000000000000000000000000000000 P f04cc9de19e842fdbfe8b42a03561182 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f04cc9de19e842fdbfe8b42a03561182" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 46863 } }
I20260926 06:20:17.202920 10405 leader_election.cc:302] T 00000000000000000000000000000000 P f04cc9de19e842fdbfe8b42a03561182 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f04cc9de19e842fdbfe8b42a03561182; no voters:
I20260926 06:20:17.203024 4906 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:20:17.203065 10405 leader_election.cc:288] T 00000000000000000000000000000000 P f04cc9de19e842fdbfe8b42a03561182 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:20:17.203125 10410 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f04cc9de19e842fdbfe8b42a03561182 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:20:17.203287 10410 raft_consensus.cc:686] T 00000000000000000000000000000000 P f04cc9de19e842fdbfe8b42a03561182 [term 1 LEADER]: Becoming Leader. State: Replica: f04cc9de19e842fdbfe8b42a03561182, State: Running, Role: LEADER
I20260926 06:20:17.203346 10405 sys_catalog.cc:564] T 00000000000000000000000000000000 P f04cc9de19e842fdbfe8b42a03561182 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:20:17.203367 10410 consensus_queue.cc:237] T 00000000000000000000000000000000 P f04cc9de19e842fdbfe8b42a03561182 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f04cc9de19e842fdbfe8b42a03561182" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 46863 } }
I20260926 06:20:17.203670 10410 sys_catalog.cc:455] T 00000000000000000000000000000000 P f04cc9de19e842fdbfe8b42a03561182 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f04cc9de19e842fdbfe8b42a03561182. Latest consensus state: current_term: 1 leader_uuid: "f04cc9de19e842fdbfe8b42a03561182" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f04cc9de19e842fdbfe8b42a03561182" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 46863 } } }
I20260926 06:20:17.203759 10410 sys_catalog.cc:458] T 00000000000000000000000000000000 P f04cc9de19e842fdbfe8b42a03561182 [sys.catalog]: This master's current role is: LEADER
I20260926 06:20:17.203681 10412 sys_catalog.cc:455] T 00000000000000000000000000000000 P f04cc9de19e842fdbfe8b42a03561182 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f04cc9de19e842fdbfe8b42a03561182" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f04cc9de19e842fdbfe8b42a03561182" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 46863 } } }
I20260926 06:20:17.203850 10412 sys_catalog.cc:458] T 00000000000000000000000000000000 P f04cc9de19e842fdbfe8b42a03561182 [sys.catalog]: This master's current role is: LEADER
I20260926 06:20:17.204778 10416 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:20:17.205219 10416 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:20:17.206631 10416 catalog_manager.cc:1347] Generated new cluster ID: e7c7a43465514b4a94afa7e1ce814ee4
I20260926 06:20:17.206704 10416 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:20:17.213771 10416 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:20:17.214273 10416 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:20:17.217456 10416 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f04cc9de19e842fdbfe8b42a03561182: Generated new TSK 0
I20260926 06:20:17.217629 10416 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:20:17.263895 10408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:17.224684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35936 (local address 127.4.202.190:46863)
0926 06:20:17.224736 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:17.224739 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:17.224884 (+ 145us) server_negotiation.cc:407] Connection header received
0926 06:20:17.224944 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:17.224945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:17.224954 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:17.225010 (+ 56us) server_negotiation.cc:226] Negotiated authn=JWT
0926 06:20:17.225159 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:17.225638 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:17.226785 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:17.226973 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:17.227028 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:17.227258 (+ 230us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 06:20:17.263625 (+ 36367us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0926 06:20:17.263819 (+ 194us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35936: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":26}
W20260926 06:20:17.263976 10408 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35936: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260926 06:20:17.263947 10434 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:20:17.224575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:46863 (local address 127.0.0.1:35936)
0926 06:20:17.224808 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:17.224819 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:17.224881 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:17.225030 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:17.225031 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:17.225033 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0926 06:20:17.225097 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:17.225099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:17.225734 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:17.225735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:17.226707 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:17.226710 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:17.227069 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:17.227070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:17.227130 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:17.227136 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 06:20:17.263761 (+ 36625us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0926 06:20:17.263873 (+ 112us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.202.190:46863: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":102,"threads_started":1}
I20260926 06:20:17.264272 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 10338
2026-09-26T06:20:17Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (215 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (85 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (84 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:20:17 dist-test-slave-pd0t krb5kdc[10446](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:20:17 dist-test-slave-pd0t krb5kdc[10446](info): set up 2 sockets
Sep 26 06:20:17 dist-test-slave-pd0t krb5kdc[10446](info): commencing operation
krb5kdc: starting...
W20260926 06:20:18.454836 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:20:18 dist-test-slave-pd0t krb5kdc[10446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:20:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:20:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.202.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
I20260926 06:20:18.567780 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:42945
--webserver_interface=127.4.202.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:39861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:42945
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:18.648082 10459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:18.648267 10459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:18.648288 10459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:18.649407 10459 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:20:18.649447 10459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:18.649461 10459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:18.649474 10459 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:20:18.649487 10459 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:20:18.651024 10459 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:39861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:42945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:42945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:18.651247 10459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:18.651473 10459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:18.656020 10467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:18.656034 10465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:18.656193 10459 server_base.cc:1034] running on GCE node
W20260926 06:20:18.656029 10464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:18.656515 10459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:18.656781 10459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:18.657918 10459 hybrid_clock.cc:648] HybridClock initialized: now 1790403618657906 us; error 31 us; skew 500 ppm
Sep 26 06:20:18 dist-test-slave-pd0t krb5kdc[10446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403618, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:18.660351 10459 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:20:18.660794 10459 webserver.cc:460] Webserver started at http://127.4.202.190:40727/ using document root <none> and password file <none>
I20260926 06:20:18.661010 10459 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:18.661052 10459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:18.661185 10459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:18.662217 10459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "8502589aa86a4d77a354d5113f7ab0f8"
format_stamp: "Formatted at 2026-09-26 06:20:18 on dist-test-slave-pd0t"
server_key: "a7c9b695db972b12baffc5f1823d98da"
server_key_iv: "a0294b2e3f4e435c0f670ed42d48486c"
server_key_version: "encryptionkey@0"
I20260926 06:20:18.662551 10459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "8502589aa86a4d77a354d5113f7ab0f8"
format_stamp: "Formatted at 2026-09-26 06:20:18 on dist-test-slave-pd0t"
server_key: "a7c9b695db972b12baffc5f1823d98da"
server_key_iv: "a0294b2e3f4e435c0f670ed42d48486c"
server_key_version: "encryptionkey@0"
I20260926 06:20:18.666910 10459 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.000s
I20260926 06:20:18.667639 10474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:18.667752 10459 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:20:18.667809 10459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "8502589aa86a4d77a354d5113f7ab0f8"
format_stamp: "Formatted at 2026-09-26 06:20:18 on dist-test-slave-pd0t"
server_key: "a7c9b695db972b12baffc5f1823d98da"
server_key_iv: "a0294b2e3f4e435c0f670ed42d48486c"
server_key_version: "encryptionkey@0"
I20260926 06:20:18.667862 10459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:18.699077 10459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:18.701213 10459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:18.701313 10459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:18.704985 10459 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:42945
I20260926 06:20:18.705036 10526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:42945 every 8 connection(s)
I20260926 06:20:18.705361 10459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:20:18.706211 10527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:18.708391 10527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8502589aa86a4d77a354d5113f7ab0f8: Bootstrap starting.
I20260926 06:20:18.708942 10527 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8502589aa86a4d77a354d5113f7ab0f8: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:18.709213 10527 log.cc:824] T 00000000000000000000000000000000 P 8502589aa86a4d77a354d5113f7ab0f8: Log is configured to *not* fsync() on all Append() calls
I20260926 06:20:18.709893 10527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8502589aa86a4d77a354d5113f7ab0f8: No bootstrap required, opened a new log
I20260926 06:20:18.711509 10527 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8502589aa86a4d77a354d5113f7ab0f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8502589aa86a4d77a354d5113f7ab0f8" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42945 } }
I20260926 06:20:18.711664 10527 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8502589aa86a4d77a354d5113f7ab0f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:18.711709 10527 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8502589aa86a4d77a354d5113f7ab0f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8502589aa86a4d77a354d5113f7ab0f8, State: Initialized, Role: FOLLOWER
I20260926 06:20:18.711900 10527 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8502589aa86a4d77a354d5113f7ab0f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8502589aa86a4d77a354d5113f7ab0f8" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42945 } }
I20260926 06:20:18.711998 10527 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8502589aa86a4d77a354d5113f7ab0f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:20:18.712034 10527 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8502589aa86a4d77a354d5113f7ab0f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:20:18.712041 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 10459
I20260926 06:20:18.712082 10527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8502589aa86a4d77a354d5113f7ab0f8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:18.712154 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:20:18.712329 4906 external_mini_cluster.cc:1428] Setting key 8de39cbff1bd013890d5efdba817b2f0
I20260926 06:20:18.712766 10527 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8502589aa86a4d77a354d5113f7ab0f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8502589aa86a4d77a354d5113f7ab0f8" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42945 } }
I20260926 06:20:18.712870 10527 leader_election.cc:302] T 00000000000000000000000000000000 P 8502589aa86a4d77a354d5113f7ab0f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8502589aa86a4d77a354d5113f7ab0f8; no voters:
I20260926 06:20:18.713009 10527 leader_election.cc:288] T 00000000000000000000000000000000 P 8502589aa86a4d77a354d5113f7ab0f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:20:18.713042 10532 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8502589aa86a4d77a354d5113f7ab0f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:20:18.713181 10532 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8502589aa86a4d77a354d5113f7ab0f8 [term 1 LEADER]: Becoming Leader. State: Replica: 8502589aa86a4d77a354d5113f7ab0f8, State: Running, Role: LEADER
I20260926 06:20:18.713294 10527 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8502589aa86a4d77a354d5113f7ab0f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:20:18.713310 10532 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8502589aa86a4d77a354d5113f7ab0f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8502589aa86a4d77a354d5113f7ab0f8" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42945 } }
I20260926 06:20:18.713654 10532 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8502589aa86a4d77a354d5113f7ab0f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8502589aa86a4d77a354d5113f7ab0f8. Latest consensus state: current_term: 1 leader_uuid: "8502589aa86a4d77a354d5113f7ab0f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8502589aa86a4d77a354d5113f7ab0f8" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42945 } } }
I20260926 06:20:18.713750 10532 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8502589aa86a4d77a354d5113f7ab0f8 [sys.catalog]: This master's current role is: LEADER
I20260926 06:20:18.713666 10533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8502589aa86a4d77a354d5113f7ab0f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8502589aa86a4d77a354d5113f7ab0f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8502589aa86a4d77a354d5113f7ab0f8" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42945 } } }
I20260926 06:20:18.713892 10533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8502589aa86a4d77a354d5113f7ab0f8 [sys.catalog]: This master's current role is: LEADER
I20260926 06:20:18.713946 10536 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:20:18.714443 10536 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:20:18.716539 10536 catalog_manager.cc:1347] Generated new cluster ID: 8e1dd287bd1640d689041d02f6a3e8f0
I20260926 06:20:18.716621 10536 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:20:18 dist-test-slave-pd0t krb5kdc[10446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:20:18.723184 10531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:18.712788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37342 (local address 127.4.202.190:42945)
0926 06:20:18.712991 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:18.713001 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:18.713052 (+ 51us) server_negotiation.cc:407] Connection header received
0926 06:20:18.713203 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:18.713211 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:18.713278 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:18.714257 (+ 979us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:18.714346 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:18.715715 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:18.716506 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:18.717107 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:18.717237 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:18.719810 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:18.719813 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:18.719813 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:18.719824 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:18.721625 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:18.722018 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:18.722021 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:18.722021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:18.722122 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:18.722330 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:18.722330 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:18.722331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:18.722618 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:18.722770 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:18.722873 (+ 103us) server_negotiation.cc:299] Negotiation successful
0926 06:20:18.722990 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":74,"threads_started":1}
I20260926 06:20:18.728412 10536 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:20:18.728885 10536 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:20:18.733071 10536 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8502589aa86a4d77a354d5113f7ab0f8: Generated new TSK 0
I20260926 06:20:18.733235 10536 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.202.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
I20260926 06:20:18.759069 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.129:0
--local_ip_for_outbound_sockets=127.4.202.129
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:42945
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:39861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:18.837679 10553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:18.837865 10553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:18.837888 10553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:18.839020 10553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:18.839059 10553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:18.839085 10553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.129
I20260926 06:20:18.840435 10553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:39861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:42945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.129
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:18.840600 10553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:18.840749 10553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:18.845393 10561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:18.845393 10559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:18.845396 10558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:18.845659 10553 server_base.cc:1034] running on GCE node
I20260926 06:20:18.845880 10553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:18.846057 10553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:18.847239 10553 hybrid_clock.cc:648] HybridClock initialized: now 1790403618847222 us; error 30 us; skew 500 ppm
Sep 26 06:20:18 dist-test-slave-pd0t krb5kdc[10446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403618, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:18.849583 10553 init.cc:377] Logged in from keytab as kudu/127.4.202.129@KRBTEST.COM (short username kudu)
I20260926 06:20:18.850025 10553 webserver.cc:460] Webserver started at http://127.4.202.129:41395/ using document root <none> and password file <none>
I20260926 06:20:18.850241 10553 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:18.850283 10553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:18.850365 10553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:18.851353 10553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-0/data/instance:
uuid: "44ff5ab84fa844c9a320b18c92d36f1e"
format_stamp: "Formatted at 2026-09-26 06:20:18 on dist-test-slave-pd0t"
server_key: "74e9367c8bb5766180c857aa38a16132"
server_key_iv: "a2dbe4ac1e703f3969321add26c42990"
server_key_version: "encryptionkey@0"
I20260926 06:20:18.851624 10553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance:
uuid: "44ff5ab84fa844c9a320b18c92d36f1e"
format_stamp: "Formatted at 2026-09-26 06:20:18 on dist-test-slave-pd0t"
server_key: "74e9367c8bb5766180c857aa38a16132"
server_key_iv: "a2dbe4ac1e703f3969321add26c42990"
server_key_version: "encryptionkey@0"
I20260926 06:20:18.852810 10553 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 06:20:18.853474 10568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:18.853646 10553 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:20:18.853730 10553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-0/wal
uuid: "44ff5ab84fa844c9a320b18c92d36f1e"
format_stamp: "Formatted at 2026-09-26 06:20:18 on dist-test-slave-pd0t"
server_key: "74e9367c8bb5766180c857aa38a16132"
server_key_iv: "a2dbe4ac1e703f3969321add26c42990"
server_key_version: "encryptionkey@0"
I20260926 06:20:18.853788 10553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:18.871912 10553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:18.874965 10553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:18.875128 10553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:18.875365 10553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:20:18.875744 10553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:20:18.875792 10553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:18.875844 10553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:20:18.875869 10553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:18.883446 10553 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.129:36071
I20260926 06:20:18.883497 10681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.129:36071 every 8 connection(s)
I20260926 06:20:18.883819 10553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
Sep 26 06:20:18 dist-test-slave-pd0t krb5kdc[10446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403618, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:20:18.893380 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 10553
I20260926 06:20:18.893481 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance
I20260926 06:20:18.893669 4906 external_mini_cluster.cc:1428] Setting key 5ec31c56a19f5c4baae27d80128b4b18
I20260926 06:20:18.895691 10531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:18.884665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:35733 (local address 127.4.202.190:42945)
0926 06:20:18.884728 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:18.884730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:18.884766 (+ 36us) server_negotiation.cc:407] Connection header received
0926 06:20:18.886187 (+ 1421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:18.886189 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:18.886202 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:18.886301 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:18.887020 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:18.887560 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:18.888628 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:18.888964 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:18.889061 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:18.891842 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:18.891843 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:18.891844 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:18.891851 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:18.893642 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:18.894476 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:18.894478 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:18.894478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:18.894543 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:18.894873 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:18.894875 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:18.894875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:18.895047 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:18.895133 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:18.895463 (+ 330us) server_negotiation.cc:299] Negotiation successful
0926 06:20:18.895559 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 06:20:18.896090 10684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:18.884467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42945 (local address 127.4.202.129:35733)
0926 06:20:18.884639 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:18.884685 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:18.886052 (+ 1367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:18.886383 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:18.886387 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:18.886736 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:18.886921 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:18.886927 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:18.887674 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:18.887675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:18.888531 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:18.888533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:18.889073 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:18.889074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:18.889185 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:18.889706 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:18.889716 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:18.891695 (+ 1979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:18.893785 (+ 2090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:18.893791 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:18.893793 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:18.894402 (+ 609us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:18.894643 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:18.894644 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:18.894644 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:18.894790 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:18.895129 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:18.895135 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:18.895305 (+ 170us) client_negotiation.cc:769] Sending connection context
0926 06:20:18.895418 (+ 113us) client_negotiation.cc:240] Negotiation successful
0926 06:20:18.895592 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":75,"threads_started":1}
I20260926 06:20:18.896677 10682 heartbeater.cc:347] Connected to a master server at 127.4.202.190:42945
I20260926 06:20:18.896790 10682 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:18.896987 10682 heartbeater.cc:511] Master 127.4.202.190:42945 requested a full tablet report, sending...
I20260926 06:20:18.897542 10491 ts_manager.cc:194] Registered new tserver with Master: 44ff5ab84fa844c9a320b18c92d36f1e (127.4.202.129:36071)
I20260926 06:20:18.898542 10491 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.129@KRBTEST.COM'} at 127.4.202.129:35733
WARNING: no policy specified for kudu/127.4.202.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
I20260926 06:20:18.921312 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.130:0
--local_ip_for_outbound_sockets=127.4.202.130
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:42945
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:39861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:18.999562 10687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:18.999737 10687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:18.999759 10687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:19.000895 10687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:19.000947 10687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:19.000973 10687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.130
I20260926 06:20:19.002462 10687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:39861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:42945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.130
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:19.002709 10687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:19.002904 10687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:19.007647 10695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:19.007644 10693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:19.007645 10692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:19.007917 10687 server_base.cc:1034] running on GCE node
I20260926 06:20:19.008131 10687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:19.008374 10687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:19.009545 10687 hybrid_clock.cc:648] HybridClock initialized: now 1790403619009536 us; error 35 us; skew 500 ppm
Sep 26 06:20:19 dist-test-slave-pd0t krb5kdc[10446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403619, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:19.011934 10687 init.cc:377] Logged in from keytab as kudu/127.4.202.130@KRBTEST.COM (short username kudu)
I20260926 06:20:19.012395 10687 webserver.cc:460] Webserver started at http://127.4.202.130:39529/ using document root <none> and password file <none>
I20260926 06:20:19.012604 10687 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:19.012717 10687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:19.012812 10687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:19.013789 10687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-1/data/instance:
uuid: "c53d81dae6bb4bd2840012d07911ec0c"
format_stamp: "Formatted at 2026-09-26 06:20:19 on dist-test-slave-pd0t"
server_key: "ba06852cbaa6a0cc4a2c8a8eb678a50b"
server_key_iv: "fe08a0614cc79ff11f3387ce223168e7"
server_key_version: "encryptionkey@0"
I20260926 06:20:19.014109 10687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance:
uuid: "c53d81dae6bb4bd2840012d07911ec0c"
format_stamp: "Formatted at 2026-09-26 06:20:19 on dist-test-slave-pd0t"
server_key: "ba06852cbaa6a0cc4a2c8a8eb678a50b"
server_key_iv: "fe08a0614cc79ff11f3387ce223168e7"
server_key_version: "encryptionkey@0"
I20260926 06:20:19.015458 10687 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:20:19.016130 10702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:19.016296 10687 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:20:19.016378 10687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-1/wal
uuid: "c53d81dae6bb4bd2840012d07911ec0c"
format_stamp: "Formatted at 2026-09-26 06:20:19 on dist-test-slave-pd0t"
server_key: "ba06852cbaa6a0cc4a2c8a8eb678a50b"
server_key_iv: "fe08a0614cc79ff11f3387ce223168e7"
server_key_version: "encryptionkey@0"
I20260926 06:20:19.016449 10687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:19.034940 10687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:19.036990 10687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:19.037138 10687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:19.037348 10687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:20:19.037629 10687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:20:19.037662 10687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:19.037681 10687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:20:19.037727 10687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:19.043311 10687 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.130:40365
I20260926 06:20:19.043372 10815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.130:40365 every 8 connection(s)
I20260926 06:20:19.043654 10687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
I20260926 06:20:19.045645 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 10687
I20260926 06:20:19.045722 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance
I20260926 06:20:19.045903 4906 external_mini_cluster.cc:1428] Setting key 902caf06908c8ae66006a0a49c528f21
Sep 26 06:20:19 dist-test-slave-pd0t krb5kdc[10446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403619, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:20:19.053759 10531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:19.044344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:40951 (local address 127.4.202.190:42945)
0926 06:20:19.044403 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:19.044406 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:19.044597 (+ 191us) server_negotiation.cc:407] Connection header received
0926 06:20:19.045417 (+ 820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:19.045418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:19.045427 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:19.045499 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:19.046271 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:19.046970 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:19.048052 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:19.048229 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:19.048287 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:19.050753 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:19.050755 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:19.050756 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:19.050763 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:19.052254 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:19.052746 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:19.052747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:19.052747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:19.052828 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:19.053123 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:19.053123 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:19.053124 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:19.053234 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:19.053305 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:19.053629 (+ 324us) server_negotiation.cc:299] Negotiation successful
0926 06:20:19.053699 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 06:20:19.054046 10818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:19.044321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42945 (local address 127.4.202.130:40951)
0926 06:20:19.044475 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:19.044512 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:19.045333 (+ 821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:19.045548 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:19.045551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:19.045859 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:19.046102 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:19.046108 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:19.047056 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:19.047057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:19.047961 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:19.047964 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:19.048344 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:19.048346 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:19.048492 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:19.048922 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:19.048929 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:19.050646 (+ 1717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:19.052341 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:19.052348 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:19.052350 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:19.052667 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:19.052903 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:19.052904 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:19.052904 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:19.053053 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:19.053320 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:19.053325 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:19.053486 (+ 161us) client_negotiation.cc:769] Sending connection context
0926 06:20:19.053618 (+ 132us) client_negotiation.cc:240] Negotiation successful
0926 06:20:19.053790 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":91,"threads_started":1}
WARNING: no policy specified for kudu/127.4.202.131@KRBTEST.COM; defaulting to no policy
I20260926 06:20:19.054639 10816 heartbeater.cc:347] Connected to a master server at 127.4.202.190:42945
I20260926 06:20:19.054752 10816 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:19.054899 10816 heartbeater.cc:511] Master 127.4.202.190:42945 requested a full tablet report, sending...
I20260926 06:20:19.055244 10491 ts_manager.cc:194] Registered new tserver with Master: c53d81dae6bb4bd2840012d07911ec0c (127.4.202.130:40365)
I20260926 06:20:19.055584 10491 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.130@KRBTEST.COM'} at 127.4.202.130:40951
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
I20260926 06:20:19.071440 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.131:0
--local_ip_for_outbound_sockets=127.4.202.131
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:42945
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:39861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:19.157081 10821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:19.157276 10821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:19.157299 10821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:19.158445 10821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:19.158484 10821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:19.158509 10821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.131
I20260926 06:20:19.160027 10821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:39861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:42945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.131
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:19.160257 10821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:19.160496 10821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:19.165367 10827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:19.165373 10826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:19.165553 10821 server_base.cc:1034] running on GCE node
W20260926 06:20:19.165469 10829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:19.165839 10821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:19.166085 10821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:19.167265 10821 hybrid_clock.cc:648] HybridClock initialized: now 1790403619167261 us; error 34 us; skew 500 ppm
Sep 26 06:20:19 dist-test-slave-pd0t krb5kdc[10446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403619, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:19.169726 10821 init.cc:377] Logged in from keytab as kudu/127.4.202.131@KRBTEST.COM (short username kudu)
I20260926 06:20:19.170070 10821 webserver.cc:460] Webserver started at http://127.4.202.131:41121/ using document root <none> and password file <none>
I20260926 06:20:19.170253 10821 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:19.170291 10821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:19.170372 10821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:19.171470 10821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-2/data/instance:
uuid: "d03242cb5247456aa76e31f1f21de84d"
format_stamp: "Formatted at 2026-09-26 06:20:19 on dist-test-slave-pd0t"
server_key: "f44e949c59d75c5f8bec8b299ac8efd1"
server_key_iv: "98ec9a9ff2901b262565b83f0935908f"
server_key_version: "encryptionkey@0"
I20260926 06:20:19.171762 10821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance:
uuid: "d03242cb5247456aa76e31f1f21de84d"
format_stamp: "Formatted at 2026-09-26 06:20:19 on dist-test-slave-pd0t"
server_key: "f44e949c59d75c5f8bec8b299ac8efd1"
server_key_iv: "98ec9a9ff2901b262565b83f0935908f"
server_key_version: "encryptionkey@0"
I20260926 06:20:19.173101 10821 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 06:20:19.173786 10836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:19.173950 10821 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:20:19.174032 10821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-2/wal
uuid: "d03242cb5247456aa76e31f1f21de84d"
format_stamp: "Formatted at 2026-09-26 06:20:19 on dist-test-slave-pd0t"
server_key: "f44e949c59d75c5f8bec8b299ac8efd1"
server_key_iv: "98ec9a9ff2901b262565b83f0935908f"
server_key_version: "encryptionkey@0"
I20260926 06:20:19.174081 10821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:19.190079 10821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:19.191838 10821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:19.191941 10821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:19.192091 10821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:20:19.192446 10821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:20:19.192479 10821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:19.192499 10821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:20:19.192512 10821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:19.197930 10821 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.131:33429
I20260926 06:20:19.198000 10949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.131:33429 every 8 connection(s)
I20260926 06:20:19.198287 10821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
Sep 26 06:20:19 dist-test-slave-pd0t krb5kdc[10446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403619, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:20:19.206233 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 10821
I20260926 06:20:19.206339 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance
I20260926 06:20:19.206507 4906 external_mini_cluster.cc:1428] Setting key de64beb673fd7675a1c6a103b0e2c5fb
I20260926 06:20:19.207358 10531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:19.198975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:60755 (local address 127.4.202.190:42945)
0926 06:20:19.199014 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:19.199016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:19.199237 (+ 221us) server_negotiation.cc:407] Connection header received
0926 06:20:19.200095 (+ 858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:19.200096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:19.200105 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:19.200158 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:19.200744 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:19.201195 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:19.202112 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:19.202346 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:19.202403 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:19.204523 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:19.204524 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:19.204524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:19.204531 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:19.205714 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:19.206208 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:19.206209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:19.206209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:19.206264 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:19.206595 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:19.206596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:19.206596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:19.206758 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:19.206831 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:19.207198 (+ 367us) server_negotiation.cc:299] Negotiation successful
0926 06:20:19.207277 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 06:20:19.207505 10952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:19.198947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42945 (local address 127.4.202.131:60755)
0926 06:20:19.199101 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:19.199139 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:19.200011 (+ 872us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:19.200217 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:19.200220 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:19.200477 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:19.200675 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:19.200678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:19.201267 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:19.201268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:19.202002 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:19.202004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:19.202417 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:19.202418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:19.202515 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:19.203028 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:19.203035 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:19.204414 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:19.205795 (+ 1381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:19.205799 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:19.205801 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:19.206092 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:19.206358 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:19.206359 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:19.206359 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:19.206499 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:19.206836 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:19.206842 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:19.206996 (+ 154us) client_negotiation.cc:769] Sending connection context
0926 06:20:19.207134 (+ 138us) client_negotiation.cc:240] Negotiation successful
0926 06:20:19.207290 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":90,"threads_started":1}
I20260926 06:20:19.207916 10950 heartbeater.cc:347] Connected to a master server at 127.4.202.190:42945
I20260926 06:20:19.208014 10950 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:19.208173 10950 heartbeater.cc:511] Master 127.4.202.190:42945 requested a full tablet report, sending...
I20260926 06:20:19.208557 10491 ts_manager.cc:194] Registered new tserver with Master: d03242cb5247456aa76e31f1f21de84d (127.4.202.131:33429)
I20260926 06:20:19.208889 10491 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.131@KRBTEST.COM'} at 127.4.202.131:60755
I20260926 06:20:19.218266 4906 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/test-admin.keytab.
Sep 26 06:20:19 dist-test-slave-pd0t krb5kdc[10446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:19.234903 4906 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260926 06:20:19.236447 10957 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:20:19.235622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38852 (local address 127.4.202.129:36071)
0926 06:20:19.235793 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:19.235797 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:19.235848 (+ 51us) server_negotiation.cc:407] Connection header received
0926 06:20:19.235894 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:19.235895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:19.235909 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:19.235971 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:19.236389 (+ 418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38852: BlockingRecv error: recv got EOF from 127.0.0.1:38852 (error 108)
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":84,"threads_started":1}
W20260926 06:20:19.236593 4906 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/krb5cc' of type FILE
Sep 26 06:20:19 dist-test-slave-pd0t krb5kdc[10446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:19.237553 4906 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 06:20:19 dist-test-slave-pd0t krb5kdc[10446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:19.238463 4906 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 06:20:19 dist-test-slave-pd0t krb5kdc[10446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.129@KRBTEST.COM
I20260926 06:20:19.245726 10957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:19.238991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38858 (local address 127.4.202.129:36071)
0926 06:20:19.239048 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:19.239050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:19.239080 (+ 30us) server_negotiation.cc:407] Connection header received
0926 06:20:19.239140 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:19.239141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:19.239149 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:19.239203 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:19.239551 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:19.240245 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:19.240933 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:19.241253 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:19.241311 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:19.243321 (+ 2010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:19.243329 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:19.243329 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:19.243335 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:19.244697 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:19.245036 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:19.245039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:19.245039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:19.245087 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:19.245248 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:19.245248 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:19.245248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:19.245359 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:19.245450 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:19.245513 (+ 63us) server_negotiation.cc:299] Negotiation successful
0926 06:20:19.245568 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260926 06:20:19.247520 10957 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:20:19.247015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38868 (local address 127.4.202.129:36071)
0926 06:20:19.247048 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:19.247049 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:19.247091 (+ 42us) server_negotiation.cc:407] Connection header received
0926 06:20:19.247148 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:19.247149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:19.247156 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:19.247204 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:19.247490 (+ 286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38868: BlockingRecv error: recv got EOF from 127.0.0.1:38868 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260926 06:20:19.247584 4906 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/krb5cc' of type FILE
Sep 26 06:20:19 dist-test-slave-pd0t krb5kdc[10446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:19.248469 4906 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 06:20:19 dist-test-slave-pd0t krb5kdc[10446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:19.249270 4906 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 06:20:19 dist-test-slave-pd0t krb5kdc[10446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.129@KRBTEST.COM
I20260926 06:20:19.256064 10957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:19.249709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38874 (local address 127.4.202.129:36071)
0926 06:20:19.249775 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:19.249777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:19.249828 (+ 51us) server_negotiation.cc:407] Connection header received
0926 06:20:19.249862 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:19.249863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:19.249869 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:19.249912 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:19.250483 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:19.251007 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:19.251759 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:19.251940 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:19.251986 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:19.253814 (+ 1828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:19.253815 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:19.253816 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:19.253821 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:19.255037 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:19.255336 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:19.255337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:19.255337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:19.255384 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:19.255540 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:19.255540 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:19.255540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:19.255668 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:19.255747 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:19.255858 (+ 111us) server_negotiation.cc:299] Negotiation successful
0926 06:20:19.255912 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260926 06:20:19.257669 10957 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:20:19.257135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38882 (local address 127.4.202.129:36071)
0926 06:20:19.257185 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:19.257188 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:19.257258 (+ 70us) server_negotiation.cc:407] Connection header received
0926 06:20:19.257324 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:19.257325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:19.257334 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:19.257384 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:19.257629 (+ 245us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38882: BlockingRecv error: recv got EOF from 127.0.0.1:38882 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260926 06:20:19.257843 4906 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790403591640726-4906-0/krb5kdc/krb5cc)
Sep 26 06:20:19 dist-test-slave-pd0t krb5kdc[10446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:19.258801 4906 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 06:20:19 dist-test-slave-pd0t krb5kdc[10446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:19.259639 4906 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 06:20:19 dist-test-slave-pd0t krb5kdc[10446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.129@KRBTEST.COM
I20260926 06:20:19.265786 10957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:19.260130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38892 (local address 127.4.202.129:36071)
0926 06:20:19.260184 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:19.260186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:19.260226 (+ 40us) server_negotiation.cc:407] Connection header received
0926 06:20:19.260293 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:19.260293 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:19.260300 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:19.260347 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:19.260672 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:19.261084 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:19.261726 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:19.261889 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:19.261937 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:19.263776 (+ 1839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:19.263777 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:19.263777 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:19.263783 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:19.265018 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:19.265299 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:19.265300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:19.265300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:19.265343 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:19.265481 (+ 138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:19.265481 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:19.265481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:19.265549 (+ 68us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:19.265607 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:19.265692 (+ 85us) server_negotiation.cc:299] Negotiation successful
0926 06:20:19.265739 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 06:20:19.266302 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 10553
I20260926 06:20:19.270784 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 10687
I20260926 06:20:19.274749 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 10821
I20260926 06:20:19.278784 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 10459
2026-09-26T06:20:19Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1847 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:20:19 dist-test-slave-pd0t krb5kdc[10970](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:20:19 dist-test-slave-pd0t krb5kdc[10970](info): set up 2 sockets
Sep 26 06:20:19 dist-test-slave-pd0t krb5kdc[10970](info): commencing operation
krb5kdc: starting...
W20260926 06:20:20.306821 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:20:20 dist-test-slave-pd0t krb5kdc[10970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:20:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:20:20Z Disabled control of system clock
WARNING: no policy specified for oryx/127.4.202.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.202.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.202.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.190.keytab.
Entry for principal oryx/127.4.202.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.190.keytab.
I20260926 06:20:20.417611 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:34455
--webserver_interface=127.4.202.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.190.keytab
--principal=oryx/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:46033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:34455
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:20.497213 10983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:20.497429 10983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:20.497465 10983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:20.498632 10983 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:20:20.498703 10983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:20.498728 10983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:20.498755 10983 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:20:20.498781 10983 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:20:20.500202 10983 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:46033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:34455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.190.keytab
--principal=oryx/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:34455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:20.500460 10983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:20.500676 10983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:20.505424 10991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:20.505553 10989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:20.505601 10983 server_base.cc:1034] running on GCE node
W20260926 06:20:20.505445 10988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:20.505878 10983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:20.506160 10983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:20.507323 10983 hybrid_clock.cc:648] HybridClock initialized: now 1790403620507297 us; error 38 us; skew 500 ppm
Sep 26 06:20:20 dist-test-slave-pd0t krb5kdc[10970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403620, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:20.516378 10983 init.cc:377] Logged in from keytab as oryx/127.4.202.190@KRBTEST.COM (short username oryx)
I20260926 06:20:20.516841 10983 webserver.cc:460] Webserver started at http://127.4.202.190:36091/ using document root <none> and password file <none>
I20260926 06:20:20.517066 10983 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:20.517112 10983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:20.517266 10983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:20.518285 10983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "7490610226774824aaae08390ef4c1e1"
format_stamp: "Formatted at 2026-09-26 06:20:20 on dist-test-slave-pd0t"
server_key: "ed19e354d8bb258455aeec0a49d1980a"
server_key_iv: "35a7a616cd1a3b9e42b975551a7962db"
server_key_version: "encryptionkey@0"
I20260926 06:20:20.518707 10983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "7490610226774824aaae08390ef4c1e1"
format_stamp: "Formatted at 2026-09-26 06:20:20 on dist-test-slave-pd0t"
server_key: "ed19e354d8bb258455aeec0a49d1980a"
server_key_iv: "35a7a616cd1a3b9e42b975551a7962db"
server_key_version: "encryptionkey@0"
I20260926 06:20:20.519992 10983 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:20:20.520700 10998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:20.520916 10983 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:20:20.521019 10983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "7490610226774824aaae08390ef4c1e1"
format_stamp: "Formatted at 2026-09-26 06:20:20 on dist-test-slave-pd0t"
server_key: "ed19e354d8bb258455aeec0a49d1980a"
server_key_iv: "35a7a616cd1a3b9e42b975551a7962db"
server_key_version: "encryptionkey@0"
I20260926 06:20:20.521097 10983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:20.535003 10983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:20.537201 10983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:20.537358 10983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:20.541245 10983 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:34455
I20260926 06:20:20.541278 11050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:34455 every 8 connection(s)
I20260926 06:20:20.541594 10983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:20:20.541741 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 10983
I20260926 06:20:20.541831 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:20:20.542047 4906 external_mini_cluster.cc:1428] Setting key c733c97ef2910fae7f84c62063fbb220
I20260926 06:20:20.542867 11051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:20.545635 11051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1: Bootstrap starting.
I20260926 06:20:20.546298 11051 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:20.546617 11051 log.cc:824] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1: Log is configured to *not* fsync() on all Append() calls
I20260926 06:20:20.547434 11051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1: No bootstrap required, opened a new log
Sep 26 06:20:20 dist-test-slave-pd0t krb5kdc[10970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM
I20260926 06:20:20.549539 11051 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } }
I20260926 06:20:20.549710 11051 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:20.549760 11051 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7490610226774824aaae08390ef4c1e1, State: Initialized, Role: FOLLOWER
I20260926 06:20:20.549917 11051 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } }
I20260926 06:20:20.550004 11051 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:20:20.550043 11051 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:20:20.550086 11051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:20.550822 11051 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } }
I20260926 06:20:20.550945 11051 leader_election.cc:302] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7490610226774824aaae08390ef4c1e1; no voters:
I20260926 06:20:20.551092 11051 leader_election.cc:288] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:20:20.551280 11057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:20:20.551332 11051 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:20:20.551731 11057 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [term 1 LEADER]: Becoming Leader. State: Replica: 7490610226774824aaae08390ef4c1e1, State: Running, Role: LEADER
I20260926 06:20:20.551923 11057 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } }
I20260926 06:20:20.552518 11057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7490610226774824aaae08390ef4c1e1. Latest consensus state: current_term: 1 leader_uuid: "7490610226774824aaae08390ef4c1e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } }
I20260926 06:20:20.552729 11057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [sys.catalog]: This master's current role is: LEADER
I20260926 06:20:20.552783 11056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7490610226774824aaae08390ef4c1e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } }
I20260926 06:20:20.553129 11056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [sys.catalog]: This master's current role is: LEADER
I20260926 06:20:20.553589 11054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:20.542620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39552 (local address 127.4.202.190:34455)
0926 06:20:20.542799 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:20.542811 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:20.542835 (+ 24us) server_negotiation.cc:407] Connection header received
0926 06:20:20.543295 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:20.543301 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:20.543347 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:20.543455 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:20.544210 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:20.545081 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:20.546128 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:20.546627 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:20.546765 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:20.549204 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:20.549208 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:20.549208 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:20.549218 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:20.551398 (+ 2180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:20.551831 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:20.551835 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:20.551835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:20.551942 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:20.552197 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:20.552198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:20.552198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:20.552518 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:20.552671 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:20.553194 (+ 523us) server_negotiation.cc:299] Negotiation successful
0926 06:20:20.553348 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":94,"threads_started":1}
W20260926 06:20:20.553982 11070 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:20:20.554034 11070 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:20:20.554111 11071 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:20:20.554247 11071 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:20:20.555823 11071 catalog_manager.cc:1347] Generated new cluster ID: 43975ab56aab4c96b64749e23928b1b4
I20260926 06:20:20.555938 11071 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:20:20.577324 11071 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:20:20.578547 11071 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:20:20.590458 11071 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1: Generated new TSK 0
I20260926 06:20:20.590637 11071 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.4.202.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.202.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.202.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.129.keytab.
Entry for principal oryx/127.4.202.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.129.keytab.
I20260926 06:20:20.623528 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.129:0
--local_ip_for_outbound_sockets=127.4.202.129
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:34455
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.129.keytab
--principal=oryx/127.4.202.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:46033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:20.701768 11077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:20.701951 11077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:20.701974 11077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:20.703181 11077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:20.703226 11077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:20.703253 11077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.129
I20260926 06:20:20.704656 11077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:46033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.129.keytab
--principal=oryx/127.4.202.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:34455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.129
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:20.704890 11077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:20.705129 11077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:20.709980 11083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:20.709980 11085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:20.710032 11082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:20.710084 11077 server_base.cc:1034] running on GCE node
I20260926 06:20:20.710393 11077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:20.710628 11077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:20.711800 11077 hybrid_clock.cc:648] HybridClock initialized: now 1790403620711787 us; error 25 us; skew 500 ppm
Sep 26 06:20:20 dist-test-slave-pd0t krb5kdc[10970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403620, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:20.714341 11077 init.cc:377] Logged in from keytab as oryx/127.4.202.129@KRBTEST.COM (short username oryx)
I20260926 06:20:20.714804 11077 webserver.cc:460] Webserver started at http://127.4.202.129:43687/ using document root <none> and password file <none>
I20260926 06:20:20.715029 11077 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:20.715095 11077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:20.715248 11077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:20.716221 11077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-0/data/instance:
uuid: "d47009f3689a4abfbbcd6623405bdcad"
format_stamp: "Formatted at 2026-09-26 06:20:20 on dist-test-slave-pd0t"
server_key: "d989e39fd2cba02c4bb9ed0bc010132f"
server_key_iv: "dc7efd6189b593c9036d931d7b1c3218"
server_key_version: "encryptionkey@0"
I20260926 06:20:20.716570 11077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance:
uuid: "d47009f3689a4abfbbcd6623405bdcad"
format_stamp: "Formatted at 2026-09-26 06:20:20 on dist-test-slave-pd0t"
server_key: "d989e39fd2cba02c4bb9ed0bc010132f"
server_key_iv: "dc7efd6189b593c9036d931d7b1c3218"
server_key_version: "encryptionkey@0"
I20260926 06:20:20.717929 11077 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:20:20.718704 11092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:20.718878 11077 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:20:20.718971 11077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-0/wal
uuid: "d47009f3689a4abfbbcd6623405bdcad"
format_stamp: "Formatted at 2026-09-26 06:20:20 on dist-test-slave-pd0t"
server_key: "d989e39fd2cba02c4bb9ed0bc010132f"
server_key_iv: "dc7efd6189b593c9036d931d7b1c3218"
server_key_version: "encryptionkey@0"
I20260926 06:20:20.719053 11077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:20.725909 11077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:20.728209 11077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:20.728366 11077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:20.728575 11077 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:20:20.728856 11077 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:20:20.728909 11077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:20.728941 11077 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:20:20.728967 11077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:20.734437 11077 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.129:37589
I20260926 06:20:20.734531 11205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.129:37589 every 8 connection(s)
I20260926 06:20:20.734859 11077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
I20260926 06:20:20.737588 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 11077
I20260926 06:20:20.737654 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance
I20260926 06:20:20.737838 4906 external_mini_cluster.cc:1428] Setting key f3a3c9b5f8e18a066193c721ea3a3905
Sep 26 06:20:20 dist-test-slave-pd0t krb5kdc[10970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403620, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM
I20260926 06:20:20.744056 11054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:20.735548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:55733 (local address 127.4.202.190:34455)
0926 06:20:20.735591 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:20.735593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:20.735798 (+ 205us) server_negotiation.cc:407] Connection header received
0926 06:20:20.736216 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:20.736218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:20.736228 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:20.736279 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:20.737126 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:20.737670 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:20.738915 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:20.739130 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:20.739220 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:20.741224 (+ 2004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:20.741225 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:20.741226 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:20.741233 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:20.742687 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:20.743092 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:20.743094 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:20.743094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:20.743149 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:20.743423 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:20.743426 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:20.743426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:20.743538 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:20.743607 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:20.743885 (+ 278us) server_negotiation.cc:299] Negotiation successful
0926 06:20:20.743968 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:20:20.744217 11208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:20.735534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.129:55733)
0926 06:20:20.735688 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:20.735720 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:20.736113 (+ 393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:20.736333 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:20.736335 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:20.736882 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:20.737050 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:20.737054 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:20.737757 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:20.737758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:20.738817 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:20.738820 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:20.739231 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:20.739232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:20.739357 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:20.739832 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:20.739840 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:20.741111 (+ 1271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:20.742776 (+ 1665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:20.742783 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:20.742785 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:20.743001 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:20.743229 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:20.743230 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:20.743230 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:20.743348 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:20.743612 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:20.743626 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:20.743755 (+ 129us) client_negotiation.cc:769] Sending connection context
0926 06:20:20.743854 (+ 99us) client_negotiation.cc:240] Negotiation successful
0926 06:20:20.743999 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":88,"threads_started":1}
I20260926 06:20:20.744791 11206 heartbeater.cc:347] Connected to a master server at 127.4.202.190:34455
I20260926 06:20:20.744875 11206 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:20.745050 11206 heartbeater.cc:511] Master 127.4.202.190:34455 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.4.202.130@KRBTEST.COM; defaulting to no policy
I20260926 06:20:20.745581 11015 ts_manager.cc:194] Registered new tserver with Master: d47009f3689a4abfbbcd6623405bdcad (127.4.202.129:37589)
I20260926 06:20:20.746157 11015 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.202.129@KRBTEST.COM'} at 127.4.202.129:55733
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.202.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.202.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.130.keytab.
Entry for principal oryx/127.4.202.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.130.keytab.
I20260926 06:20:20.761191 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.130:0
--local_ip_for_outbound_sockets=127.4.202.130
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:34455
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.130.keytab
--principal=oryx/127.4.202.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:46033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:20.839077 11211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:20.839262 11211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:20.839283 11211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:20.840299 11211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:20.840332 11211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:20.840358 11211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.130
I20260926 06:20:20.841635 11211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:46033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.130.keytab
--principal=oryx/127.4.202.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:34455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.130
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:20.841790 11211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:20.841941 11211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:20.846681 11219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:20.846709 11217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:20.846779 11211 server_base.cc:1034] running on GCE node
W20260926 06:20:20.846730 11216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:20.847038 11211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:20.847285 11211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:20.848443 11211 hybrid_clock.cc:648] HybridClock initialized: now 1790403620848432 us; error 40 us; skew 500 ppm
Sep 26 06:20:20 dist-test-slave-pd0t krb5kdc[10970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403620, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:20.850826 11211 init.cc:377] Logged in from keytab as oryx/127.4.202.130@KRBTEST.COM (short username oryx)
I20260926 06:20:20.851329 11211 webserver.cc:460] Webserver started at http://127.4.202.130:36517/ using document root <none> and password file <none>
I20260926 06:20:20.851555 11211 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:20.851621 11211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:20.851755 11211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:20.852828 11211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-1/data/instance:
uuid: "20ac61fdecdf433ab60c8eb9b7f678cc"
format_stamp: "Formatted at 2026-09-26 06:20:20 on dist-test-slave-pd0t"
server_key: "f37d079f46c4ff890600f9a215087bb8"
server_key_iv: "0f1c96b1eaa0aff270744755b9164333"
server_key_version: "encryptionkey@0"
I20260926 06:20:20.853204 11211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance:
uuid: "20ac61fdecdf433ab60c8eb9b7f678cc"
format_stamp: "Formatted at 2026-09-26 06:20:20 on dist-test-slave-pd0t"
server_key: "f37d079f46c4ff890600f9a215087bb8"
server_key_iv: "0f1c96b1eaa0aff270744755b9164333"
server_key_version: "encryptionkey@0"
I20260926 06:20:20.854539 11211 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:20:20.855394 11226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:20.855695 11211 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:20:20.855836 11211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-1/wal
uuid: "20ac61fdecdf433ab60c8eb9b7f678cc"
format_stamp: "Formatted at 2026-09-26 06:20:20 on dist-test-slave-pd0t"
server_key: "f37d079f46c4ff890600f9a215087bb8"
server_key_iv: "0f1c96b1eaa0aff270744755b9164333"
server_key_version: "encryptionkey@0"
I20260926 06:20:20.855929 11211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:20.870774 11211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:20.872752 11211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:20.872925 11211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:20.873184 11211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:20:20.873535 11211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:20:20.873593 11211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:20.873638 11211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:20:20.873675 11211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:20.879963 11211 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.130:41657
I20260926 06:20:20.880019 11339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.130:41657 every 8 connection(s)
I20260926 06:20:20.880357 11211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
I20260926 06:20:20.885586 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 11211
I20260926 06:20:20.885716 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance
I20260926 06:20:20.885896 4906 external_mini_cluster.cc:1428] Setting key d9572db56ceed5a32c2ad3883f225192
Sep 26 06:20:20 dist-test-slave-pd0t krb5kdc[10970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403620, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM
I20260926 06:20:20.890064 11054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:20.881043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:56309 (local address 127.4.202.190:34455)
0926 06:20:20.881115 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:20.881117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:20.881306 (+ 189us) server_negotiation.cc:407] Connection header received
0926 06:20:20.881784 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:20.881785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:20.881794 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:20.881846 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:20.882823 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:20.883301 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:20.884325 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:20.884512 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:20.884563 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:20.886813 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:20.886815 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:20.886816 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:20.886823 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:20.888489 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:20.888999 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:20.888999 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:20.889000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:20.889049 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:20.889355 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:20.889356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:20.889357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:20.889508 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:20.889587 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:20.889838 (+ 251us) server_negotiation.cc:299] Negotiation successful
0926 06:20:20.889924 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 06:20:20.890074 11342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:20.881000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.130:56309)
0926 06:20:20.881160 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:20.881204 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:20.881697 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:20.881911 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:20.881914 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:20.882505 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:20.882735 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:20.882740 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:20.883384 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:20.883385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:20.884242 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:20.884244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:20.884592 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:20.884593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:20.884700 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:20.885138 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:20.885146 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:20.886706 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:20.888575 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:20.888581 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:20.888583 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:20.888923 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:20.889115 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:20.889116 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:20.889116 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:20.889266 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:20.889581 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:20.889586 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:20.889700 (+ 114us) client_negotiation.cc:769] Sending connection context
0926 06:20:20.889791 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 06:20:20.889912 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":91,"threads_started":1}
I20260926 06:20:20.890754 11340 heartbeater.cc:347] Connected to a master server at 127.4.202.190:34455
I20260926 06:20:20.890864 11340 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:20.891062 11340 heartbeater.cc:511] Master 127.4.202.190:34455 requested a full tablet report, sending...
I20260926 06:20:20.891450 11015 ts_manager.cc:194] Registered new tserver with Master: 20ac61fdecdf433ab60c8eb9b7f678cc (127.4.202.130:41657)
I20260926 06:20:20.891914 11015 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.202.130@KRBTEST.COM'} at 127.4.202.130:56309
WARNING: no policy specified for oryx/127.4.202.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.202.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.202.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.131.keytab.
Entry for principal oryx/127.4.202.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.131.keytab.
I20260926 06:20:20.911726 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.131:0
--local_ip_for_outbound_sockets=127.4.202.131
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:34455
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.131.keytab
--principal=oryx/127.4.202.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:46033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:20.990995 11345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:20.991179 11345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:20.991209 11345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:20.992291 11345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:20.992326 11345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:20.992352 11345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.131
I20260926 06:20:20.993691 11345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:46033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.131.keytab
--principal=oryx/127.4.202.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:34455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.131
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:20.993855 11345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:20.994014 11345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:20.998814 11353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:20.998858 11345 server_base.cc:1034] running on GCE node
W20260926 06:20:20.998831 11351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:20.998822 11350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:20.999172 11345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:20.999387 11345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:21.000522 11345 hybrid_clock.cc:648] HybridClock initialized: now 1790403621000517 us; error 32 us; skew 500 ppm
Sep 26 06:20:20 dist-test-slave-pd0t krb5kdc[10970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403620, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:21.002960 11345 init.cc:377] Logged in from keytab as oryx/127.4.202.131@KRBTEST.COM (short username oryx)
I20260926 06:20:21.003407 11345 webserver.cc:460] Webserver started at http://127.4.202.131:36607/ using document root <none> and password file <none>
I20260926 06:20:21.003612 11345 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:21.003655 11345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:21.003782 11345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:21.004761 11345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-2/data/instance:
uuid: "a640f2f67fae4b4fb555f3ae032d02e4"
format_stamp: "Formatted at 2026-09-26 06:20:21 on dist-test-slave-pd0t"
server_key: "06a6e104136ba77923fabb901a466ceb"
server_key_iv: "79cdf5adcb46e15a03565fe1f226cb39"
server_key_version: "encryptionkey@0"
I20260926 06:20:21.005113 11345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance:
uuid: "a640f2f67fae4b4fb555f3ae032d02e4"
format_stamp: "Formatted at 2026-09-26 06:20:21 on dist-test-slave-pd0t"
server_key: "06a6e104136ba77923fabb901a466ceb"
server_key_iv: "79cdf5adcb46e15a03565fe1f226cb39"
server_key_version: "encryptionkey@0"
I20260926 06:20:21.006389 11345 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 06:20:21.007225 11360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:21.007412 11345 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 06:20:21.007481 11345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-2/wal
uuid: "a640f2f67fae4b4fb555f3ae032d02e4"
format_stamp: "Formatted at 2026-09-26 06:20:21 on dist-test-slave-pd0t"
server_key: "06a6e104136ba77923fabb901a466ceb"
server_key_iv: "79cdf5adcb46e15a03565fe1f226cb39"
server_key_version: "encryptionkey@0"
I20260926 06:20:21.007560 11345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:21.021548 11345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:21.023612 11345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:21.023764 11345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:21.023967 11345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:20:21.024259 11345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:20:21.024314 11345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:21.024353 11345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:20:21.024380 11345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:21.030120 11345 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.131:38777
I20260926 06:20:21.030161 11473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.131:38777 every 8 connection(s)
I20260926 06:20:21.030460 11345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
I20260926 06:20:21.035975 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 11345
I20260926 06:20:21.036084 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance
I20260926 06:20:21.036293 4906 external_mini_cluster.cc:1428] Setting key 2c8ccb2e39418d5309d091ba306c46c1
Sep 26 06:20:21 dist-test-slave-pd0t krb5kdc[10970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403620, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM
I20260926 06:20:21.040052 11054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.031104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:48919 (local address 127.4.202.190:34455)
0926 06:20:21.031143 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.031144 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.031391 (+ 247us) server_negotiation.cc:407] Connection header received
0926 06:20:21.031890 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.031891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.031902 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.031955 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:21.032873 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.033368 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.034354 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.034558 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.034683 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.037105 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:21.037106 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:21.037106 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:21.037114 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:21.038704 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.039106 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.039107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.039107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.039167 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.039430 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.039431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.039431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.039548 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:21.039624 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:21.039889 (+ 265us) server_negotiation.cc:299] Negotiation successful
0926 06:20:21.039964 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 06:20:21.040246 11476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.031105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.131:48919)
0926 06:20:21.031276 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:21.031308 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:21.031788 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:21.032021 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:21.032025 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:21.032588 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:21.032771 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.032775 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.033462 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.033463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.034246 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.034248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.034699 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.034700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.034816 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.035284 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:21.035292 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:21.037013 (+ 1721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:21.038794 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.038801 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.038803 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.039016 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.039249 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.039250 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.039251 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.039361 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.039616 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:21.039621 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:21.039760 (+ 139us) client_negotiation.cc:769] Sending connection context
0926 06:20:21.039868 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 06:20:21.040014 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":85,"threads_started":1}
I20260926 06:20:21.040655 11474 heartbeater.cc:347] Connected to a master server at 127.4.202.190:34455
I20260926 06:20:21.040745 11474 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:21.040870 11474 heartbeater.cc:511] Master 127.4.202.190:34455 requested a full tablet report, sending...
I20260926 06:20:21.041155 11015 ts_manager.cc:194] Registered new tserver with Master: a640f2f67fae4b4fb555f3ae032d02e4 (127.4.202.131:38777)
I20260926 06:20:21.041468 11015 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.202.131@KRBTEST.COM'} at 127.4.202.131:48919
I20260926 06:20:21.047976 4906 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.4.202.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.202.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.202.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.189.keytab.
Entry for principal oryx/127.4.202.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.189.keytab.
I20260926 06:20:21.073173 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.189:40149
--webserver_interface=127.4.202.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.189.keytab
--principal=oryx/127.4.202.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:46033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:34455,127.4.202.189:40149
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:21.151978 11479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:21.152206 11479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:21.152248 11479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:21.153419 11479 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:20:21.153465 11479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:21.153479 11479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:21.153491 11479 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:20:21.153542 11479 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:20:21.155036 11479 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:46033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:34455,127.4.202.189:40149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.189.keytab
--principal=oryx/127.4.202.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.189:40149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.202.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
W20260926 06:20:21.155126 11479 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.4.202.190:34455,127.4.202.189:40149'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260926 06:20:21.155278 11479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:21.155499 11479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:21.160318 11484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:21.160297 11485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:21.160336 11479 server_base.cc:1034] running on GCE node
W20260926 06:20:21.160305 11487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:21.160688 11479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:21.160974 11479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:21.162137 11479 hybrid_clock.cc:648] HybridClock initialized: now 1790403621162120 us; error 33 us; skew 500 ppm
Sep 26 06:20:21 dist-test-slave-pd0t krb5kdc[10970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403621, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:21.164600 11479 init.cc:377] Logged in from keytab as oryx/127.4.202.189@KRBTEST.COM (short username oryx)
I20260926 06:20:21.164939 11479 webserver.cc:460] Webserver started at http://127.4.202.189:46307/ using document root <none> and password file <none>
I20260926 06:20:21.165115 11479 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:21.165154 11479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:21.165242 11479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:21.166169 11479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-1/data/instance:
uuid: "1245bf7b57c549d28ef3f35463029b01"
format_stamp: "Formatted at 2026-09-26 06:20:21 on dist-test-slave-pd0t"
server_key: "357674b22b0d80e6bf0d83f8511f1de9"
server_key_iv: "b3a0d11ee0e2f97b5a835c4c8f06fe86"
server_key_version: "encryptionkey@0"
I20260926 06:20:21.166462 11479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-1/wal/instance:
uuid: "1245bf7b57c549d28ef3f35463029b01"
format_stamp: "Formatted at 2026-09-26 06:20:21 on dist-test-slave-pd0t"
server_key: "357674b22b0d80e6bf0d83f8511f1de9"
server_key_iv: "b3a0d11ee0e2f97b5a835c4c8f06fe86"
server_key_version: "encryptionkey@0"
I20260926 06:20:21.167852 11479 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:20:21.168584 11494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:21.168777 11479 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:20:21.168869 11479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-1/wal
uuid: "1245bf7b57c549d28ef3f35463029b01"
format_stamp: "Formatted at 2026-09-26 06:20:21 on dist-test-slave-pd0t"
server_key: "357674b22b0d80e6bf0d83f8511f1de9"
server_key_iv: "b3a0d11ee0e2f97b5a835c4c8f06fe86"
server_key_version: "encryptionkey@0"
I20260926 06:20:21.168948 11479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:21.192328 11479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:21.194664 11479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:21.194820 11479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:21.198966 11479 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.189:40149
I20260926 06:20:21.199000 11546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.189:40149 every 8 connection(s)
I20260926 06:20:21.199359 11479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-1/data/info.pb
I20260926 06:20:21.200321 11547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:21.201766 11547 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 06:20:21 dist-test-slave-pd0t krb5kdc[10970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403621, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.189@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM
I20260926 06:20:21.207538 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 11479
I20260926 06:20:21.211140 11054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.202537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39560 (local address 127.4.202.190:34455)
0926 06:20:21.202671 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.202673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.202813 (+ 140us) server_negotiation.cc:407] Connection header received
0926 06:20:21.203247 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.203248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.203258 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.203317 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:21.203945 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.204435 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.205417 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.205606 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.205655 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.207982 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:21.207983 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:21.207984 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:21.207991 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:21.209707 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.210124 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.210126 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.210126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.210201 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.210449 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.210450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.210450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.210640 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:21.210723 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:21.210980 (+ 257us) server_negotiation.cc:299] Negotiation successful
0926 06:20:21.211054 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260926 06:20:21.211325 11549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.202369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.0.0.1:39560)
0926 06:20:21.202655 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:21.202710 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:21.203152 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:21.203381 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:21.203385 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:21.203647 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:21.203872 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.203876 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.204533 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.204534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.205335 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.205337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.205674 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.205675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.205770 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.206280 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:21.206287 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:21.207883 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:21.209791 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.209796 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.209798 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.209999 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.210266 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.210267 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.210267 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.210376 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.210715 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:21.210720 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:21.210853 (+ 133us) client_negotiation.cc:769] Sending connection context
0926 06:20:21.210960 (+ 107us) client_negotiation.cc:240] Negotiation successful
0926 06:20:21.211107 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":100,"threads_started":1}
I20260926 06:20:21.212035 11547 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.202.189" port: 40149 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 06:20:21 dist-test-slave-pd0t krb5kdc[10970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.202.189@KRBTEST.COM
Sep 26 06:20:21 dist-test-slave-pd0t krb5kdc[10970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403621, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.189@KRBTEST.COM for oryx/127.4.202.189@KRBTEST.COM
I20260926 06:20:21.216984 11551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.208024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36722 (local address 127.4.202.189:40149)
0926 06:20:21.208173 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.208177 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.208264 (+ 87us) server_negotiation.cc:407] Connection header received
0926 06:20:21.208288 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.208291 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.208303 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.208372 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:21.209115 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.210130 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.210877 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.211374 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.211438 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.214172 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:21.214178 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:21.214179 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:21.214185 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:21.215722 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.216083 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.216086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.216086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.216149 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.216345 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.216346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.216346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.216561 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:21.216736 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:21.216784 (+ 48us) server_negotiation.cc:299] Negotiation successful
0926 06:20:21.216868 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":67,"threads_started":1}
I20260926 06:20:21.219008 11549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.212336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:40149 (local address 127.0.0.1:36732)
0926 06:20:21.212379 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:21.212390 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:21.212434 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:21.212724 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:21.212724 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:21.212826 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:21.212866 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.212867 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.213588 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.213589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.214134 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.214136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.214529 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.214530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.214621 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.214946 (+ 325us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:21.214949 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:21.215959 (+ 1010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:21.218101 (+ 2142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.218102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.218102 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.218277 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.218446 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.218447 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.218447 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.218491 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.218797 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:21.218798 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:21.218830 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 06:20:21.218882 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 06:20:21.218955 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260926 06:20:21.219108 11552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.212397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36732 (local address 127.4.202.189:40149)
0926 06:20:21.212581 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.212583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.212592 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:20:21.212620 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.212621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.212631 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.212715 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:21.212931 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.213508 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.214235 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.214448 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.214530 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.216052 (+ 1522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:21.216053 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:21.216054 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:21.216059 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:21.218034 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.218341 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.218341 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.218342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.218390 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.218559 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.218560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.218560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.218733 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:21.218793 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:21.218915 (+ 122us) server_negotiation.cc:299] Negotiation successful
0926 06:20:21.218999 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":162,"thread_start_us":120,"threads_started":1}
I20260926 06:20:21.220273 11547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01: Bootstrap starting.
I20260926 06:20:21.220806 11547 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:21.221019 11547 log.cc:824] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01: Log is configured to *not* fsync() on all Append() calls
I20260926 06:20:21.221680 11547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01: No bootstrap required, opened a new log
I20260926 06:20:21.223271 11547 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 40149 } }
I20260926 06:20:21.223387 11547 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:21.223407 11547 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1245bf7b57c549d28ef3f35463029b01, State: Initialized, Role: FOLLOWER
I20260926 06:20:21.223519 11547 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 40149 } }
I20260926 06:20:21.223762 11554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 40149 } } }
I20260926 06:20:21.223841 11554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:20:21.223848 11547 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [sys.catalog]: configured and running, proceeding with master startup.
W20260926 06:20:21.225291 11565 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:20:21.225349 11565 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:20:21.226847 11479 master_runner.cc:428] Detected that this master 1245bf7b57c549d28ef3f35463029b01 is joining an existing cluster
I20260926 06:20:21.226920 11479 master_runner.cc:432] Initiating AddMaster RPC to add 127.4.202.189:40149
I20260926 06:20:21.235544 11552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.227851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36740 (local address 127.4.202.189:40149)
0926 06:20:21.227887 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.227889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.228129 (+ 240us) server_negotiation.cc:407] Connection header received
0926 06:20:21.228201 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.228202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.228210 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.228256 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:21.228721 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.229435 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.230500 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.230801 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.230914 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.231915 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:21.231916 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:21.231916 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:21.231922 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:21.233637 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.234031 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.234032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.234032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.234091 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.234971 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.234972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.234973 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.235093 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:21.235162 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:21.235339 (+ 177us) server_negotiation.cc:299] Negotiation successful
0926 06:20:21.235456 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 06:20:21.235841 11572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.227955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:40149 (local address 127.0.0.1:36740)
0926 06:20:21.228067 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:21.228077 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:21.228149 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:21.228262 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:21.228263 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:21.228561 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:21.228624 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.228625 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.229515 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.229516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.230404 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.230406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.230913 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.230914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.230971 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.231411 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:21.231419 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:21.231819 (+ 400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:21.233736 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.233737 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.233738 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.233947 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.234155 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.234156 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.234157 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.234244 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.235165 (+ 921us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:21.235165 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:21.235208 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 06:20:21.235617 (+ 409us) client_negotiation.cc:240] Negotiation successful
0926 06:20:21.235723 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":76,"threads_started":1}
I20260926 06:20:21.236105 11571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.227824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.0.0.1:39566)
0926 06:20:21.227975 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:21.227986 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:21.228066 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:21.228283 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:21.228284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:21.228545 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:21.228593 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.228594 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.230518 (+ 1924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.230518 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.231366 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.231368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.231855 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.231856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.231918 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.232336 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:21.232341 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:21.232736 (+ 395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:21.234922 (+ 2186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.234923 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.234923 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.235136 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.235387 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.235388 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.235388 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.235449 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.235822 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:21.235823 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:21.235863 (+ 40us) client_negotiation.cc:769] Sending connection context
0926 06:20:21.235928 (+ 65us) client_negotiation.cc:240] Negotiation successful
0926 06:20:21.236019 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":106,"threads_started":1}
I20260926 06:20:21.236228 11054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.227898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39566 (local address 127.4.202.190:34455)
0926 06:20:21.227946 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.227948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.228041 (+ 93us) server_negotiation.cc:407] Connection header received
0926 06:20:21.228135 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.228136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.228147 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.228201 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:21.228668 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.229427 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.231453 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.231759 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.231832 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.232811 (+ 979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:21.232812 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:21.232813 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:21.232820 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:21.234851 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.235259 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.235260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.235260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.235318 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.235538 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.235538 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.235539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.235693 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:21.235760 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:21.235968 (+ 208us) server_negotiation.cc:299] Negotiation successful
0926 06:20:21.236036 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
Sep 26 06:20:21 dist-test-slave-pd0t krb5kdc[10970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403620, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.190@KRBTEST.COM for oryx/127.4.202.189@KRBTEST.COM
I20260926 06:20:21.246867 11552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.238525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36742 (local address 127.4.202.189:40149)
0926 06:20:21.238562 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.238564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.238721 (+ 157us) server_negotiation.cc:407] Connection header received
0926 06:20:21.238818 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.238819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.238828 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.238871 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:21.239245 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.239712 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.240449 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.240647 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.240726 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.243602 (+ 2876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:21.243605 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:21.243605 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:21.243620 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:21.245321 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.245812 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.245813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.245813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.245898 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.246241 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.246242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.246243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.246399 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:21.246439 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:21.246708 (+ 269us) server_negotiation.cc:299] Negotiation successful
0926 06:20:21.246784 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:20:21.247153 11573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.238469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:40149 (local address 127.0.0.1:36742)
0926 06:20:21.238606 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:21.238659 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:21.238745 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:21.238874 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:21.238877 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:21.239126 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:21.239168 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.239169 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.239802 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.239803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.240357 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.240358 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.240743 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.240744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.240818 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.241643 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:21.241648 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:21.243470 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:21.245417 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.245424 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.245426 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.245708 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.246007 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.246008 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.246008 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.246102 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.246487 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:21.246494 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:21.246542 (+ 48us) client_negotiation.cc:769] Sending connection context
0926 06:20:21.246936 (+ 394us) client_negotiation.cc:240] Negotiation successful
0926 06:20:21.247054 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":62,"threads_started":1}
I20260926 06:20:21.249056 11015 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.4.202.189:40149: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "1245bf7b57c549d28ef3f35463029b01"
member_type: NON_VOTER
last_known_addr {
host: "127.4.202.189"
port: 40149
}
attrs {
promote: true
}
}
dest_uuid: "7490610226774824aaae08390ef4c1e1"
cas_config_opid_index: -1
I20260926 06:20:21.249374 11015 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: NON_VOTER last_known_addr { host: "127.4.202.189" port: 40149 } attrs { promote: true } }
I20260926 06:20:21.250016 11574 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 1245bf7b57c549d28ef3f35463029b01 (127.4.202.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: NON_VOTER last_known_addr { host: "127.4.202.189" port: 40149 } attrs { promote: true } } }
I20260926 06:20:21.250176 11521 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:21.251142 11521 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 1 FOLLOWER]: Refusing update from remote peer 7490610226774824aaae08390ef4c1e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260926 06:20:21.251199 11574 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.4.202.189:40149
I20260926 06:20:21.251394 11479 master.cc:537] Master@127.4.202.189:40149 shutting down...
I20260926 06:20:21.251879 11576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7490610226774824aaae08390ef4c1e1" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: NON_VOTER last_known_addr { host: "127.4.202.189" port: 40149 } attrs { promote: true } } }
I20260926 06:20:21.252014 11575 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: NON_VOTER last_known_addr { host: "127.4.202.189" port: 40149 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:20:21.252094 11576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [sys.catalog]: This master's current role is: LEADER
W20260926 06:20:21.252851 11000 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260926 06:20:21.253767 11000 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 -> Peer 1245bf7b57c549d28ef3f35463029b01 (127.4.202.189:40149): Couldn't send request to peer 1245bf7b57c549d28ef3f35463029b01. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260926 06:20:21.254325 11479 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:20:21.254408 11479 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:20:21.254459 11479 tablet_replica.cc:330] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01: stopping tablet replica
I20260926 06:20:21.267251 11479 master.cc:552] Master@127.4.202.189:40149 shutdown complete.
I20260926 06:20:21.267309 11479 master_runner.cc:305] Clearing existing system tablet
I20260926 06:20:21.267424 11479 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:21.267480 11479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:21.268090 11479 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:20:21.268630 11580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:21.268781 11479 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:21.268839 11479 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:21.268862 11479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-1/wal
uuid: "1245bf7b57c549d28ef3f35463029b01"
format_stamp: "Formatted at 2026-09-26 06:20:21 on dist-test-slave-pd0t"
server_key: "357674b22b0d80e6bf0d83f8511f1de9"
server_key_iv: "b3a0d11ee0e2f97b5a835c4c8f06fe86"
server_key_version: "encryptionkey@0"
I20260926 06:20:21.269583 11479 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:20:21.273252 11479 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260926 06:20:21.273325 11479 log.cc:1196] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260926 06:20:21.273525 11479 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01: Deleting consensus metadata
I20260926 06:20:21.273653 11479 master_runner.cc:315] Copying system tablet from 127.4.202.190:34455
I20260926 06:20:21.274266 11479 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01: tablet copy: Beginning tablet copy session from remote peer at address 127.4.202.190:34455
I20260926 06:20:21.280297 11054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.274693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39572 (local address 127.4.202.190:34455)
0926 06:20:21.274737 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.274739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.274790 (+ 51us) server_negotiation.cc:407] Connection header received
0926 06:20:21.274866 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.274867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.274877 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.274961 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:21.275274 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.275753 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.276460 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.276651 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.276739 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.277684 (+ 945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:21.277685 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:21.277686 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:21.277693 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:21.279155 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.279522 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.279524 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.279525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.279581 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.279788 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.279789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.279790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.279900 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:21.279981 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:21.280131 (+ 150us) server_negotiation.cc:299] Negotiation successful
0926 06:20:21.280211 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 06:20:21.280311 11586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.274569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.0.0.1:39572)
0926 06:20:21.274735 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:21.274746 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:21.274806 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:21.274967 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:21.274969 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:21.275142 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:21.275191 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.275193 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.275831 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.275832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.276361 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.276363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.276740 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.276741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.276814 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.277290 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:21.277296 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:21.277577 (+ 281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:21.279264 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.279265 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.279266 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.279437 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.279647 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.279647 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.279648 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.279711 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.279972 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:21.279973 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:21.280017 (+ 44us) client_negotiation.cc:769] Sending connection context
0926 06:20:21.280105 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 06:20:21.280206 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":108,"threads_started":1}
I20260926 06:20:21.280779 11035 tablet_copy_service.cc:140] P 7490610226774824aaae08390ef4c1e1: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 1245bf7b57c549d28ef3f35463029b01 ({username='oryx', principal='oryx/127.4.202.189@KRBTEST.COM'} at 127.0.0.1:39572)
I20260926 06:20:21.280864 11035 tablet_copy_service.cc:161] P 7490610226774824aaae08390ef4c1e1: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 1245bf7b57c549d28ef3f35463029b01 at {username='oryx', principal='oryx/127.4.202.189@KRBTEST.COM'} at 127.0.0.1:39572: session id = 1245bf7b57c549d28ef3f35463029b01-00000000000000000000000000000000
I20260926 06:20:21.281473 11035 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 06:20:21.282073 11479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260926 06:20:21.283308 11479 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01: tablet copy: Starting download of 0 data blocks...
I20260926 06:20:21.283449 11479 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01: tablet copy: Starting download of 1 WAL segments...
I20260926 06:20:21.284272 11479 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 06:20:21.285126 11035 tablet_copy_service.cc:342] P 7490610226774824aaae08390ef4c1e1: Request end of tablet copy session 1245bf7b57c549d28ef3f35463029b01-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.4.202.189@KRBTEST.COM'} at 127.0.0.1:39572
I20260926 06:20:21.285185 11035 tablet_copy_service.cc:433] P 7490610226774824aaae08390ef4c1e1: ending tablet copy session 1245bf7b57c549d28ef3f35463029b01-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 1245bf7b57c549d28ef3f35463029b01
I20260926 06:20:21.287840 11479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:21.288937 11590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:21.288944 11587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:21.289194 11588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:21.289355 11479 server_base.cc:1034] running on GCE node
I20260926 06:20:21.289474 11479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:21.289618 11479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:21.290757 11479 hybrid_clock.cc:648] HybridClock initialized: now 1790403621290748 us; error 24 us; skew 500 ppm
Sep 26 06:20:21 dist-test-slave-pd0t krb5kdc[10970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403621, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:21.292266 11479 init.cc:377] Logged in from keytab as oryx/127.4.202.189@KRBTEST.COM (short username oryx)
I20260926 06:20:21.292516 11479 webserver.cc:460] Webserver started at http://127.4.202.189:44991/ using document root <none> and password file <none>
I20260926 06:20:21.292625 11479 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:21.292658 11479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:21.293308 11479 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:20:21.293843 11597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:21.293999 11479 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:21.294064 11479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-1/wal
uuid: "1245bf7b57c549d28ef3f35463029b01"
format_stamp: "Formatted at 2026-09-26 06:20:21 on dist-test-slave-pd0t"
server_key: "357674b22b0d80e6bf0d83f8511f1de9"
server_key_iv: "b3a0d11ee0e2f97b5a835c4c8f06fe86"
server_key_version: "encryptionkey@0"
I20260926 06:20:21.294121 11479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:21.312821 11479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:21.314317 11479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:21.316959 11479 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.189:40149
I20260926 06:20:21.316994 11649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.189:40149 every 8 connection(s)
I20260926 06:20:21.317272 11479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-1/data/info.pb
I20260926 06:20:21.317984 11650 sys_catalog.cc:263] Verifying existing consensus state
I20260926 06:20:21.318403 11650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01: Bootstrap starting.
I20260926 06:20:21.320608 11650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 06:20:21.320870 11650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01: Bootstrap complete.
I20260926 06:20:21.321036 11650 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: NON_VOTER last_known_addr { host: "127.4.202.189" port: 40149 } attrs { promote: true } }
I20260926 06:20:21.321115 11650 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1245bf7b57c549d28ef3f35463029b01, State: Initialized, Role: LEARNER
I20260926 06:20:21.321149 11650 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: NON_VOTER last_known_addr { host: "127.4.202.189" port: 40149 } attrs { promote: true } }
I20260926 06:20:21.321301 11650 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:20:21.322862 11665 catalog_manager.cc:1259] Loaded cluster ID: 43975ab56aab4c96b64749e23928b1b4
I20260926 06:20:21.322916 11665 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01: loading cluster ID for follower catalog manager: success
I20260926 06:20:21.324057 11665 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01: acquiring CA information for follower catalog manager: success
I20260926 06:20:21.324409 11665 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 06:20:21.324512 11654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: NON_VOTER last_known_addr { host: "127.4.202.189" port: 40149 } attrs { promote: true } } }
I20260926 06:20:21.324604 11654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [sys.catalog]: This master's current role is: LEARNER
I20260926 06:20:21.326617 11652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.318798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36760 (local address 127.4.202.189:40149)
0926 06:20:21.318953 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.318956 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.318964 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:20:21.318986 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.318987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.318996 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.319070 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:21.319763 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.320754 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.321809 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.322118 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.322203 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.323710 (+ 1507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:21.323711 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:21.323711 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:21.323718 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:21.325440 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.325861 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.325862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.325862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.325911 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.326107 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.326107 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.326108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.326224 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:21.326296 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:21.326451 (+ 155us) server_negotiation.cc:299] Negotiation successful
0926 06:20:21.326517 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":78,"threads_started":1}
WARNING: no policy specified for oryx/127.4.202.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.202.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.202.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.188.keytab.
Entry for principal oryx/127.4.202.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.188.keytab.
I20260926 06:20:21.355060 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.188:33663
--webserver_interface=127.4.202.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.188.keytab
--principal=oryx/127.4.202.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:46033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:34455,127.4.202.189:40149,127.4.202.188:33663
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:21.434871 11668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:21.435053 11668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:21.435074 11668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:21.436214 11668 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:20:21.436255 11668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:21.436269 11668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:21.436282 11668 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:20:21.436331 11668 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:20:21.437812 11668 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:46033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:34455,127.4.202.189:40149,127.4.202.188:33663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.188.keytab
--principal=oryx/127.4.202.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.188:33663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.202.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:21.438054 11668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:21.438278 11668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:21.442849 11674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:21.442849 11676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:21.442876 11673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:21.443184 11668 server_base.cc:1034] running on GCE node
I20260926 06:20:21.443404 11668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:21.443686 11668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:21.444833 11668 hybrid_clock.cc:648] HybridClock initialized: now 1790403621444821 us; error 30 us; skew 500 ppm
Sep 26 06:20:21 dist-test-slave-pd0t krb5kdc[10970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403621, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:21.447288 11668 init.cc:377] Logged in from keytab as oryx/127.4.202.188@KRBTEST.COM (short username oryx)
I20260926 06:20:21.447721 11668 webserver.cc:460] Webserver started at http://127.4.202.188:42543/ using document root <none> and password file <none>
I20260926 06:20:21.447894 11668 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:21.447932 11668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:21.448015 11668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:21.448944 11668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-2/data/instance:
uuid: "092902ed31314ba181f705bd4701e4c7"
format_stamp: "Formatted at 2026-09-26 06:20:21 on dist-test-slave-pd0t"
server_key: "76b22a3b42e4c2458c4f6ba4863b4012"
server_key_iv: "3a0f8c454977db5f9046bf5621cee2ce"
server_key_version: "encryptionkey@0"
I20260926 06:20:21.449294 11668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-2/wal/instance:
uuid: "092902ed31314ba181f705bd4701e4c7"
format_stamp: "Formatted at 2026-09-26 06:20:21 on dist-test-slave-pd0t"
server_key: "76b22a3b42e4c2458c4f6ba4863b4012"
server_key_iv: "3a0f8c454977db5f9046bf5621cee2ce"
server_key_version: "encryptionkey@0"
I20260926 06:20:21.450488 11668 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 06:20:21.451210 11683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:21.451419 11668 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:21.451529 11668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-2/wal
uuid: "092902ed31314ba181f705bd4701e4c7"
format_stamp: "Formatted at 2026-09-26 06:20:21 on dist-test-slave-pd0t"
server_key: "76b22a3b42e4c2458c4f6ba4863b4012"
server_key_iv: "3a0f8c454977db5f9046bf5621cee2ce"
server_key_version: "encryptionkey@0"
I20260926 06:20:21.451596 11668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:21.487128 11668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:21.489346 11668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:21.489450 11668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:21.494728 11668 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.188:33663
I20260926 06:20:21.494760 11735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.188:33663 every 8 connection(s)
I20260926 06:20:21.495132 11668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-2/data/info.pb
I20260926 06:20:21.496053 11736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:21.497568 11736 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:20:21.499796 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 11668
Sep 26 06:20:21 dist-test-slave-pd0t krb5kdc[10970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403621, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.188@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM
Sep 26 06:20:21 dist-test-slave-pd0t krb5kdc[10970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.202.188@KRBTEST.COM
I20260926 06:20:21.507772 11054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.498317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39576 (local address 127.4.202.190:34455)
0926 06:20:21.498397 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.498399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.498522 (+ 123us) server_negotiation.cc:407] Connection header received
0926 06:20:21.499039 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.499040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.499051 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.499114 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:21.500094 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.500703 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.501913 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.502250 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.502318 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.504754 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:21.504755 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:21.504756 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:21.504763 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:21.506272 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.506674 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.506675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.506676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.506732 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.507016 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.507017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.507017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.507182 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:21.507308 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:21.507574 (+ 266us) server_negotiation.cc:299] Negotiation successful
0926 06:20:21.507655 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 06:20:21.508083 11738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.498222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.0.0.1:39576)
0926 06:20:21.498390 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:21.498433 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:21.498937 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:21.499184 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:21.499187 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:21.499422 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:21.499617 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.499621 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.500806 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.500807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.501826 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.501829 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.502350 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.502352 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.502482 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.502975 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:21.502985 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:21.504650 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:21.506345 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.506352 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.506354 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.506573 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.506798 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.506799 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.506800 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.506943 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.507297 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:21.507303 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:21.507440 (+ 137us) client_negotiation.cc:769] Sending connection context
0926 06:20:21.507542 (+ 102us) client_negotiation.cc:240] Negotiation successful
0926 06:20:21.507756 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":69,"threads_started":1}
I20260926 06:20:21.508728 11736 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.202.189" port: 40149 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:20:21.509114 11739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.500315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36526 (local address 127.4.202.188:33663)
0926 06:20:21.500488 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.500492 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.500506 (+ 14us) server_negotiation.cc:407] Connection header received
0926 06:20:21.500523 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.500524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.500539 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.500622 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:21.501359 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.502190 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.503219 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.503507 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.503625 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.506187 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:21.506195 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:21.506195 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:21.506202 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:21.507794 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.508161 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.508164 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.508164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.508223 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.508549 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.508550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.508550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.508704 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:21.508853 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:21.508932 (+ 79us) server_negotiation.cc:299] Negotiation successful
0926 06:20:21.509010 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":96,"threads_started":1}
Sep 26 06:20:21 dist-test-slave-pd0t krb5kdc[10970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403621, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.188@KRBTEST.COM for oryx/127.4.202.189@KRBTEST.COM
I20260926 06:20:21.515108 11738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.508987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:40149 (local address 127.0.0.1:36764)
0926 06:20:21.509027 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:21.509037 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:21.509145 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:21.509466 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:21.509467 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:21.509570 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:21.509597 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.509598 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.510302 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.510302 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.511199 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.511202 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.511557 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.511558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.511599 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.511931 (+ 332us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:21.511935 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:21.512981 (+ 1046us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:21.514256 (+ 1275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.514257 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.514257 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.514409 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.514617 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.514618 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.514618 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.514681 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.514903 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:21.514903 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:21.514935 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 06:20:21.514985 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 06:20:21.515052 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260926 06:20:21.515116 11652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.509040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36764 (local address 127.4.202.189:40149)
0926 06:20:21.509083 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.509085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.509094 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:20:21.509373 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.509375 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.509387 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.509450 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:21.509655 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.510184 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.511299 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.511495 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.511551 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.513057 (+ 1506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:21.513058 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:21.513058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:21.513064 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:21.514183 (+ 1119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.514470 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.514471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.514471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.514522 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.514740 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.514741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.514741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.514848 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:21.514901 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:21.515000 (+ 99us) server_negotiation.cc:299] Negotiation successful
0926 06:20:21.515061 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 06:20:21.516009 11736 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.202.188" port: 33663 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 06:20:21 dist-test-slave-pd0t krb5kdc[10970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403621, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.188@KRBTEST.COM for oryx/127.4.202.188@KRBTEST.COM
I20260926 06:20:21.521734 11738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.516272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:33663 (local address 127.0.0.1:36538)
0926 06:20:21.516312 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:21.516342 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:21.516380 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:21.516499 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:21.516499 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:21.516621 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:21.516643 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.516644 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.517262 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.517262 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.517724 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.517725 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.518103 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.518104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.518147 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.518460 (+ 313us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:21.518464 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:21.519525 (+ 1061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:21.520821 (+ 1296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.520822 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.520822 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.521048 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.521270 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.521271 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.521271 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.521327 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.521528 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:21.521529 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:21.521560 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 06:20:21.521610 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 06:20:21.521680 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260926 06:20:21.521911 11739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.516313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36538 (local address 127.4.202.188:33663)
0926 06:20:21.516360 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.516362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.516370 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:20:21.516433 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.516434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.516443 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.516496 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:21.516717 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.517164 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.517813 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.518024 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.518103 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.519614 (+ 1511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:21.519615 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:21.519615 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:21.519621 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:21.520725 (+ 1104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.521129 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.521130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.521130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.521201 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.521392 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.521393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.521393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.521476 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:21.521521 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:21.521623 (+ 102us) server_negotiation.cc:299] Negotiation successful
0926 06:20:21.521795 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":33}
I20260926 06:20:21.523309 11736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7: Bootstrap starting.
I20260926 06:20:21.524084 11736 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:21.524683 11736 log.cc:824] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7: Log is configured to *not* fsync() on all Append() calls
I20260926 06:20:21.525403 11736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7: No bootstrap required, opened a new log
I20260926 06:20:21.526800 11736 raft_consensus.cc:348] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 40149 } } peers { permanent_uuid: "092902ed31314ba181f705bd4701e4c7" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 33663 } }
I20260926 06:20:21.526919 11736 raft_consensus.cc:374] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:21.526942 11736 raft_consensus.cc:729] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 092902ed31314ba181f705bd4701e4c7, State: Initialized, Role: FOLLOWER
I20260926 06:20:21.527091 11736 consensus_queue.cc:260] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 40149 } } peers { permanent_uuid: "092902ed31314ba181f705bd4701e4c7" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 33663 } }
I20260926 06:20:21.527328 11741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 40149 } } peers { permanent_uuid: "092902ed31314ba181f705bd4701e4c7" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 33663 } } }
I20260926 06:20:21.527374 11736 sys_catalog.cc:564] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:20:21.527436 11741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7 [sys.catalog]: This master's current role is: FOLLOWER
W20260926 06:20:21.528937 11752 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:20:21.529006 11752 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:20:21.531045 11668 master_runner.cc:428] Detected that this master 092902ed31314ba181f705bd4701e4c7 is joining an existing cluster
I20260926 06:20:21.531109 11668 master_runner.cc:432] Initiating AddMaster RPC to add 127.4.202.188:33663
I20260926 06:20:21.539695 11758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.532347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.0.0.1:39592)
0926 06:20:21.532488 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:21.532499 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:21.532559 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:21.532739 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:21.532740 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:21.532927 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:21.533001 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.533004 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.533923 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.533924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.534824 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.534826 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.535345 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.535346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.535417 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.535879 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:21.535883 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:21.536337 (+ 454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:21.538254 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.538255 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.538255 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.538472 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.538759 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.538760 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.538761 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.538835 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.539377 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:21.539378 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:21.539421 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 06:20:21.539497 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 06:20:21.539589 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
I20260926 06:20:21.539736 11054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.532409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39592 (local address 127.4.202.190:34455)
0926 06:20:21.532480 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.532482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.532546 (+ 64us) server_negotiation.cc:407] Connection header received
0926 06:20:21.532613 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.532614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.532626 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.532690 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:21.533089 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.533841 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.534920 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.535221 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.535260 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.536453 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:21.536455 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:21.536456 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:21.536464 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:21.538169 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.538614 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.538615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.538616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.538672 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.539148 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.539150 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.539150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.539292 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:21.539370 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:21.539540 (+ 170us) server_negotiation.cc:299] Negotiation successful
0926 06:20:21.539619 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
Sep 26 06:20:21 dist-test-slave-pd0t krb5kdc[10970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403620, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.190@KRBTEST.COM for oryx/127.4.202.188@KRBTEST.COM
I20260926 06:20:21.555438 11759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.532579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:40149 (local address 127.0.0.1:36772)
0926 06:20:21.532677 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:21.532686 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:21.532720 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:21.533948 (+ 1228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:21.533949 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:21.534145 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:21.534200 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.534201 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.535131 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.535132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.535972 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.535974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.536489 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.536490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.536551 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.537558 (+ 1007us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:21.537563 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:21.537937 (+ 374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:21.540106 (+ 2169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.540106 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.540107 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.540312 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.540538 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.540539 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.540539 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.554740 (+ 14201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.555064 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:21.555065 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:21.555115 (+ 50us) client_negotiation.cc:769] Sending connection context
0926 06:20:21.555212 (+ 97us) client_negotiation.cc:240] Negotiation successful
0926 06:20:21.555320 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":67,"threads_started":1}
I20260926 06:20:21.555461 11652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.532918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36772 (local address 127.4.202.189:40149)
0926 06:20:21.532987 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.532988 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.532996 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:20:21.533015 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.533016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.533025 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.533074 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:21.534292 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.535033 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.536072 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.536374 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.536416 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.538053 (+ 1637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:21.538054 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:21.538055 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:21.538063 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:21.539802 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.540415 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.540416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.540416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.540470 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.554835 (+ 14365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.554836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.554837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.554972 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:21.555017 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:21.555249 (+ 232us) server_negotiation.cc:299] Negotiation successful
0926 06:20:21.555344 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 06:20:21.558912 11573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.543119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:33663 (local address 127.0.0.1:36562)
0926 06:20:21.543174 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:21.543184 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:21.543224 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:21.543544 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:21.543545 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:21.543700 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:21.543739 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.543741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.544362 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.544364 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.545155 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.545157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.545486 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.545487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.545551 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.545877 (+ 326us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:21.545882 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:21.547203 (+ 1321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:21.557733 (+ 10530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.557734 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.557735 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.557960 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.558220 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.558221 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.558221 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.558308 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.558600 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:21.558601 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:21.558648 (+ 47us) client_negotiation.cc:769] Sending connection context
0926 06:20:21.558715 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 06:20:21.558814 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260926 06:20:21.558931 11739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.532950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36554 (local address 127.4.202.188:33663)
0926 06:20:21.533112 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.533113 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.533119 (+ 6us) server_negotiation.cc:407] Connection header received
0926 06:20:21.533134 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.533134 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.533141 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.533230 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:21.533533 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.534040 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.534992 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.535269 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.535304 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.555927 (+ 20623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:21.555928 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:21.555928 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:21.555934 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:21.557719 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.558045 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.558046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.558047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.558099 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.558309 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.558310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.558311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.558427 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:21.558511 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:21.558684 (+ 173us) server_negotiation.cc:299] Negotiation successful
0926 06:20:21.558768 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260926 06:20:21.559110 11760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.532749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:33663 (local address 127.0.0.1:36554)
0926 06:20:21.532846 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:21.532854 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:21.532891 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:21.533236 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:21.533237 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:21.533407 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:21.533451 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.533453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.534119 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.534119 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.534921 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.534923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.535362 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.535362 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.535417 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.555450 (+ 20033us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:21.555455 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:21.555830 (+ 375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:21.557795 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.557795 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.557796 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.557952 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.558175 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.558176 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.558176 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.558240 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.558504 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:21.558504 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:21.558541 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 06:20:21.558669 (+ 128us) client_negotiation.cc:240] Negotiation successful
0926 06:20:21.558825 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"mutex_wait_us":57,"thread_start_us":68,"threads_started":1}
I20260926 06:20:21.559279 11761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.543290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36562 (local address 127.4.202.188:33663)
0926 06:20:21.543425 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.543426 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.543433 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:20:21.543459 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.543459 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.543467 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.543519 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:21.543812 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.544275 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.545236 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.545416 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.545466 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.547310 (+ 1844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:21.547311 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:21.547312 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:21.547318 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:21.557643 (+ 10325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.558075 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.558076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.558076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.558124 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.558377 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.558377 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.558378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.558492 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:21.558606 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:21.559083 (+ 477us) server_negotiation.cc:299] Negotiation successful
0926 06:20:21.559159 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":84,"threads_started":1}
I20260926 06:20:21.559657 11015 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.4.202.188:33663: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "092902ed31314ba181f705bd4701e4c7"
member_type: NON_VOTER
last_known_addr {
host: "127.4.202.188"
port: 33663
}
attrs {
promote: true
}
}
dest_uuid: "7490610226774824aaae08390ef4c1e1"
cas_config_opid_index: 5
I20260926 06:20:21.559880 11015 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: NON_VOTER last_known_addr { host: "127.4.202.189" port: 40149 } attrs { promote: true } } peers { permanent_uuid: "092902ed31314ba181f705bd4701e4c7" member_type: NON_VOTER last_known_addr { host: "127.4.202.188" port: 33663 } attrs { promote: true } }
I20260926 06:20:21.560226 11576 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 092902ed31314ba181f705bd4701e4c7 (127.4.202.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: NON_VOTER last_known_addr { host: "127.4.202.189" port: 40149 } attrs { promote: true } } peers { permanent_uuid: "092902ed31314ba181f705bd4701e4c7" member_type: NON_VOTER last_known_addr { host: "127.4.202.188" port: 33663 } attrs { promote: true } } }
I20260926 06:20:21.560623 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 11077
I20260926 06:20:21.560698 11701 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:21.561090 11576 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.4.202.188:33663
I20260926 06:20:21.561156 11576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7490610226774824aaae08390ef4c1e1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: NON_VOTER last_known_addr { host: "127.4.202.189" port: 40149 } attrs { promote: true } } peers { permanent_uuid: "092902ed31314ba181f705bd4701e4c7" member_type: NON_VOTER last_known_addr { host: "127.4.202.188" port: 33663 } attrs { promote: true } } }
I20260926 06:20:21.561220 11576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [sys.catalog]: This master's current role is: LEADER
I20260926 06:20:21.561663 11668 master.cc:537] Master@127.4.202.188:33663 shutting down...
I20260926 06:20:21.561730 11701 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7 [term 1 FOLLOWER]: Refusing update from remote peer 7490610226774824aaae08390ef4c1e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260926 06:20:21.562731 11575 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "092902ed31314ba181f705bd4701e4c7" member_type: NON_VOTER last_known_addr { host: "127.4.202.188" port: 33663 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 06:20:21.563721 11000 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 -> Peer 092902ed31314ba181f705bd4701e4c7 (127.4.202.188:33663): Couldn't send request to peer 092902ed31314ba181f705bd4701e4c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260926 06:20:21.567248 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.129:37589
--local_ip_for_outbound_sockets=127.4.202.129
--tserver_master_addrs=127.4.202.190:34455
--webserver_port=43687
--webserver_interface=127.4.202.129
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.129.keytab
--principal=oryx/127.4.202.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:46033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.4.202.190:34455,127.4.202.189:40149
--tserver_master_addrs=127.4.202.190:34455,127.4.202.189:40149,127.4.202.188:33663 with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:20:21.567790 11668 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:20:21.567866 11668 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:20:21.567910 11668 tablet_replica.cc:330] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7: stopping tablet replica
I20260926 06:20:21.572962 11573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.560362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:40149 (local address 127.0.0.1:36784)
0926 06:20:21.562738 (+ 2376us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:21.562750 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:21.562792 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:21.569099 (+ 6307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:21.569101 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:21.569103 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:20:21.569146 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.569148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.570070 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.570072 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.571445 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.571449 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.572669 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.572671 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.572746 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.572767 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 06:20:21.572824 (+ 57us) client_negotiation.cc:240] Negotiation successful
0926 06:20:21.572868 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2360}
I20260926 06:20:21.573374 11652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.560442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36784 (local address 127.4.202.189:40149)
0926 06:20:21.560481 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.560482 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.568915 (+ 8433us) server_negotiation.cc:407] Connection header received
0926 06:20:21.568944 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.568945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.568969 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.569039 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:20:21.569233 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.569975 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.571558 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.572479 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.572560 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.572631 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:21.573237 (+ 606us) server_negotiation.cc:299] Negotiation successful
0926 06:20:21.573286 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 06:20:21.573748 11624 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 1 LEARNER]: Refusing update from remote peer 7490610226774824aaae08390ef4c1e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260926 06:20:21.574100 11575 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: NON_VOTER last_known_addr { host: "127.4.202.189" port: 40149 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260926 06:20:21.574695 11624 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 092902ed31314ba181f705bd4701e4c7 (127.4.202.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: NON_VOTER last_known_addr { host: "127.4.202.189" port: 40149 } attrs { promote: true } } peers { permanent_uuid: "092902ed31314ba181f705bd4701e4c7" member_type: NON_VOTER last_known_addr { host: "127.4.202.188" port: 33663 } attrs { promote: true } } }
I20260926 06:20:21.575188 11668 master.cc:552] Master@127.4.202.188:33663 shutdown complete.
I20260926 06:20:21.575222 11668 master_runner.cc:305] Clearing existing system tablet
I20260926 06:20:21.575364 11668 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:21.575400 11668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:21.575899 11654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7490610226774824aaae08390ef4c1e1. Latest consensus state: current_term: 1 leader_uuid: "7490610226774824aaae08390ef4c1e1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: NON_VOTER last_known_addr { host: "127.4.202.189" port: 40149 } attrs { promote: true } } peers { permanent_uuid: "092902ed31314ba181f705bd4701e4c7" member_type: NON_VOTER last_known_addr { host: "127.4.202.188" port: 33663 } attrs { promote: true } } }
I20260926 06:20:21.575997 11654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [sys.catalog]: This master's current role is: LEARNER
I20260926 06:20:21.576040 11654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7490610226774824aaae08390ef4c1e1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: NON_VOTER last_known_addr { host: "127.4.202.189" port: 40149 } attrs { promote: true } } peers { permanent_uuid: "092902ed31314ba181f705bd4701e4c7" member_type: NON_VOTER last_known_addr { host: "127.4.202.188" port: 33663 } attrs { promote: true } } }
I20260926 06:20:21.576098 11654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [sys.catalog]: This master's current role is: LEARNER
I20260926 06:20:21.576280 11575 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1: attempting to promote NON_VOTER 1245bf7b57c549d28ef3f35463029b01 to VOTER
I20260926 06:20:21.576480 11575 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 40149 } attrs { promote: false } } peers { permanent_uuid: "092902ed31314ba181f705bd4701e4c7" member_type: NON_VOTER last_known_addr { host: "127.4.202.188" port: 33663 } attrs { promote: true } }
I20260926 06:20:21.576654 11668 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:20:21.576977 11573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.576856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:33663 (local address 127.0.0.1:36574)
0926 06:20:21.576918 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:21.576952 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.188:33663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260926 06:20:21.577277 11573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.577213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:33663 (local address 127.0.0.1:36580)
0926 06:20:21.577239 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:21.577258 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.188:33663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260926 06:20:21.577335 11000 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 -> Peer 092902ed31314ba181f705bd4701e4c7 (127.4.202.188:33663): Couldn't send request to peer 092902ed31314ba181f705bd4701e4c7. Status: Network error: Client connection negotiation failed: client connection to 127.4.202.188:33663: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 06:20:21.577427 11624 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 1 LEARNER]: Refusing update from remote peer 7490610226774824aaae08390ef4c1e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260926 06:20:21.577598 11574 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 40149 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260926 06:20:21.578203 11576 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 1245bf7b57c549d28ef3f35463029b01 (127.4.202.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 40149 } attrs { promote: false } } peers { permanent_uuid: "092902ed31314ba181f705bd4701e4c7" member_type: NON_VOTER last_known_addr { host: "127.4.202.188" port: 33663 } attrs { promote: true } } }
I20260926 06:20:21.578714 11766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:21.578863 11668 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:21.578917 11668 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260926 06:20:21.578945 11668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-2/wal
uuid: "092902ed31314ba181f705bd4701e4c7"
format_stamp: "Formatted at 2026-09-26 06:20:21 on dist-test-slave-pd0t"
server_key: "76b22a3b42e4c2458c4f6ba4863b4012"
server_key_iv: "3a0f8c454977db5f9046bf5621cee2ce"
server_key_version: "encryptionkey@0"
I20260926 06:20:21.579046 11576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7490610226774824aaae08390ef4c1e1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 40149 } attrs { promote: false } } peers { permanent_uuid: "092902ed31314ba181f705bd4701e4c7" member_type: NON_VOTER last_known_addr { host: "127.4.202.188" port: 33663 } attrs { promote: true } } }
I20260926 06:20:21.579113 11576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [sys.catalog]: This master's current role is: LEADER
I20260926 06:20:21.579154 11576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7490610226774824aaae08390ef4c1e1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 40149 } attrs { promote: false } } peers { permanent_uuid: "092902ed31314ba181f705bd4701e4c7" member_type: NON_VOTER last_known_addr { host: "127.4.202.188" port: 33663 } attrs { promote: true } } }
I20260926 06:20:21.579192 11576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1 [sys.catalog]: This master's current role is: LEADER
I20260926 06:20:21.579458 11624 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 1245bf7b57c549d28ef3f35463029b01 (127.4.202.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 40149 } attrs { promote: false } } peers { permanent_uuid: "092902ed31314ba181f705bd4701e4c7" member_type: NON_VOTER last_known_addr { host: "127.4.202.188" port: 33663 } attrs { promote: true } } }
I20260926 06:20:21.579694 11668 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:20:21.580391 11764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7490610226774824aaae08390ef4c1e1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 40149 } attrs { promote: false } } peers { permanent_uuid: "092902ed31314ba181f705bd4701e4c7" member_type: NON_VOTER last_known_addr { host: "127.4.202.188" port: 33663 } attrs { promote: true } } }
I20260926 06:20:21.580469 11764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:20:21.580920 11668 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260926 06:20:21.580974 11668 log.cc:1196] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260926 06:20:21.581151 11668 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7: Deleting consensus metadata
I20260926 06:20:21.581264 11668 master_runner.cc:315] Copying system tablet from 127.4.202.190:34455
I20260926 06:20:21.581842 11668 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7: tablet copy: Beginning tablet copy session from remote peer at address 127.4.202.190:34455
I20260926 06:20:21.589309 11054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.582221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39602 (local address 127.4.202.190:34455)
0926 06:20:21.582255 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.582257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.582876 (+ 619us) server_negotiation.cc:407] Connection header received
0926 06:20:21.582902 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.582903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.582913 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.582977 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:21.583304 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.584042 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.585058 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.585360 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.585426 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.586516 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:21.586517 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:21.586518 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:21.586525 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:21.588145 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.588549 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.588550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.588550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.588607 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.588809 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.588810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.588810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.588931 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:21.588994 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:21.589145 (+ 151us) server_negotiation.cc:299] Negotiation successful
0926 06:20:21.589223 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 06:20:21.589322 11772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.582084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.0.0.1:39602)
0926 06:20:21.582711 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:21.582721 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:21.582780 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:21.582994 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:21.582995 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:21.583182 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:21.583225 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.583227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.584135 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.584136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.584973 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.584976 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.585439 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.585440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.585510 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.586040 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:21.586044 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:21.586424 (+ 380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:21.588240 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.588241 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.588241 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.588466 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.588680 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.588680 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.588681 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.588744 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.589002 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:21.589004 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:21.589043 (+ 39us) client_negotiation.cc:769] Sending connection context
0926 06:20:21.589110 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 06:20:21.589204 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":60,"threads_started":1}
I20260926 06:20:21.589840 11035 tablet_copy_service.cc:140] P 7490610226774824aaae08390ef4c1e1: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 092902ed31314ba181f705bd4701e4c7 ({username='oryx', principal='oryx/127.4.202.188@KRBTEST.COM'} at 127.0.0.1:39602)
I20260926 06:20:21.589890 11035 tablet_copy_service.cc:161] P 7490610226774824aaae08390ef4c1e1: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 092902ed31314ba181f705bd4701e4c7 at {username='oryx', principal='oryx/127.4.202.188@KRBTEST.COM'} at 127.0.0.1:39602: session id = 092902ed31314ba181f705bd4701e4c7-00000000000000000000000000000000
I20260926 06:20:21.590418 11035 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 06:20:21.590914 11668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260926 06:20:21.592067 11668 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7: tablet copy: Starting download of 0 data blocks...
I20260926 06:20:21.592295 11668 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7: tablet copy: Starting download of 1 WAL segments...
I20260926 06:20:21.593118 11668 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 06:20:21.593987 11035 tablet_copy_service.cc:342] P 7490610226774824aaae08390ef4c1e1: Request end of tablet copy session 092902ed31314ba181f705bd4701e4c7-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.4.202.188@KRBTEST.COM'} at 127.0.0.1:39602
I20260926 06:20:21.594035 11035 tablet_copy_service.cc:433] P 7490610226774824aaae08390ef4c1e1: ending tablet copy session 092902ed31314ba181f705bd4701e4c7-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 092902ed31314ba181f705bd4701e4c7
I20260926 06:20:21.596509 11668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:21.597675 11773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:21.599472 11776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:21.603503 11668 server_base.cc:1034] running on GCE node
W20260926 06:20:21.603577 11774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:21.603725 11668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:21.603856 11668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:21.607079 11668 hybrid_clock.cc:648] HybridClock initialized: now 1790403621607056 us; error 40 us; skew 500 ppm
Sep 26 06:20:21 dist-test-slave-pd0t krb5kdc[10970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403621, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:21.608563 11668 init.cc:377] Logged in from keytab as oryx/127.4.202.188@KRBTEST.COM (short username oryx)
I20260926 06:20:21.608814 11668 webserver.cc:460] Webserver started at http://127.4.202.188:38119/ using document root <none> and password file <none>
I20260926 06:20:21.608933 11668 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:21.608973 11668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:21.609699 11668 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:20:21.610265 11783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:21.610415 11668 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:21.610472 11668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-2/wal
uuid: "092902ed31314ba181f705bd4701e4c7"
format_stamp: "Formatted at 2026-09-26 06:20:21 on dist-test-slave-pd0t"
server_key: "76b22a3b42e4c2458c4f6ba4863b4012"
server_key_iv: "3a0f8c454977db5f9046bf5621cee2ce"
server_key_version: "encryptionkey@0"
I20260926 06:20:21.610528 11668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:21.620047 11668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:21.621760 11668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:21.624943 11668 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.188:33663
I20260926 06:20:21.625285 11668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/master-2/data/info.pb
I20260926 06:20:21.625555 11835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.188:33663 every 8 connection(s)
I20260926 06:20:21.626092 11836 sys_catalog.cc:263] Verifying existing consensus state
I20260926 06:20:21.626627 11836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7: Bootstrap starting.
I20260926 06:20:21.629320 11836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 06:20:21.629629 11836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7: Bootstrap complete.
I20260926 06:20:21.629863 11836 raft_consensus.cc:348] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 40149 } attrs { promote: false } } peers { permanent_uuid: "092902ed31314ba181f705bd4701e4c7" member_type: NON_VOTER last_known_addr { host: "127.4.202.188" port: 33663 } attrs { promote: true } }
I20260926 06:20:21.630007 11836 raft_consensus.cc:729] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 092902ed31314ba181f705bd4701e4c7, State: Initialized, Role: LEARNER
I20260926 06:20:21.630079 11836 consensus_queue.cc:260] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 40149 } attrs { promote: false } } peers { permanent_uuid: "092902ed31314ba181f705bd4701e4c7" member_type: NON_VOTER last_known_addr { host: "127.4.202.188" port: 33663 } attrs { promote: true } }
I20260926 06:20:21.630306 11836 sys_catalog.cc:564] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:20:21.630354 11839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 40149 } attrs { promote: false } } peers { permanent_uuid: "092902ed31314ba181f705bd4701e4c7" member_type: NON_VOTER last_known_addr { host: "127.4.202.188" port: 33663 } attrs { promote: true } } }
I20260926 06:20:21.630455 11839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7 [sys.catalog]: This master's current role is: LEARNER
I20260926 06:20:21.632083 11850 catalog_manager.cc:1259] Loaded cluster ID: 43975ab56aab4c96b64749e23928b1b4
I20260926 06:20:21.632601 11850 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7: loading cluster ID for follower catalog manager: success
I20260926 06:20:21.633666 11850 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7: acquiring CA information for follower catalog manager: success
I20260926 06:20:21.633950 11850 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260926 06:20:21.664211 11762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:21.664422 11762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:21.664459 11762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:21.665565 11762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:21.665629 11762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:21.665673 11762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.129
I20260926 06:20:21.667184 11762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:46033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.129.keytab
--principal=oryx/127.4.202.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.129:37589
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.202.129
--webserver_port=43687
--tserver_master_addrs=127.4.202.190:34455,127.4.202.189:40149,127.4.202.188:33663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.129
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:21.667428 11762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:21.667656 11762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:21.672533 11858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:21.672526 11855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:21.672528 11856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:21.673399 11762 server_base.cc:1034] running on GCE node
I20260926 06:20:21.673602 11762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:21.673844 11762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:21.674998 11762 hybrid_clock.cc:648] HybridClock initialized: now 1790403621674979 us; error 40 us; skew 500 ppm
Sep 26 06:20:21 dist-test-slave-pd0t krb5kdc[10970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403621, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:21.677500 11762 init.cc:377] Logged in from keytab as oryx/127.4.202.129@KRBTEST.COM (short username oryx)
I20260926 06:20:21.677989 11762 webserver.cc:460] Webserver started at http://127.4.202.129:43687/ using document root <none> and password file <none>
I20260926 06:20:21.678217 11762 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:21.678298 11762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:21.679662 11762 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:20:21.680251 11865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:21.680522 11762 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:20:21.680613 11762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-0/wal
uuid: "d47009f3689a4abfbbcd6623405bdcad"
format_stamp: "Formatted at 2026-09-26 06:20:20 on dist-test-slave-pd0t"
server_key: "d989e39fd2cba02c4bb9ed0bc010132f"
server_key_iv: "dc7efd6189b593c9036d931d7b1c3218"
server_key_version: "encryptionkey@0"
I20260926 06:20:21.680984 11762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:21.694536 11762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:21.696394 11762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:21.696496 11762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:21.696658 11762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:20:21.696872 11762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:20:21.696900 11762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:21.696920 11762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:20:21.696933 11762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:21.702661 11762 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.129:37589
I20260926 06:20:21.703039 11762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
I20260926 06:20:21.703127 11978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.129:37589 every 8 connection(s)
Sep 26 06:20:21 dist-test-slave-pd0t krb5kdc[10970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403621, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM
Sep 26 06:20:21 dist-test-slave-pd0t krb5kdc[10970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403621, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.188@KRBTEST.COM
Sep 26 06:20:21 dist-test-slave-pd0t krb5kdc[10970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403621, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.189@KRBTEST.COM
I20260926 06:20:21.712754 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 11762
I20260926 06:20:21.712898 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 11211
I20260926 06:20:21.713292 11054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.703952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:41703 (local address 127.4.202.190:34455)
0926 06:20:21.704006 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.704008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.704132 (+ 124us) server_negotiation.cc:407] Connection header received
0926 06:20:21.704586 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.704587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.704599 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.704666 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:21.705388 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.706143 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.707340 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.707546 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.707630 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.709757 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:21.709758 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:21.709759 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:21.709766 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:21.711675 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.712136 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.712138 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.712138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.712226 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.712509 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.712510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.712510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.712672 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:21.712746 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:21.713077 (+ 331us) server_negotiation.cc:299] Negotiation successful
0926 06:20:21.713157 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 06:20:21.713855 11983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.703698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:33363 (local address 127.4.202.188:33663)
0926 06:20:21.703794 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.703796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.703989 (+ 193us) server_negotiation.cc:407] Connection header received
0926 06:20:21.704585 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.704586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.704599 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.704668 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:21.705369 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.706029 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.707094 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.707327 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.707387 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.710285 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:21.710286 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:21.710286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:21.710293 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:21.712193 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.712596 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.712597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.712597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.712652 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.712895 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.712896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.712897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.713079 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:21.713145 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:21.713613 (+ 468us) server_negotiation.cc:299] Negotiation successful
0926 06:20:21.713710 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71,"thread_start_us":50,"threads_started":1}
I20260926 06:20:21.714365 11984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.703694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:33663 (local address 127.4.202.129:33363)
0926 06:20:21.703856 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:21.703898 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:21.704473 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:21.704738 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:21.704742 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:21.705089 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:21.705270 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.705274 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.706138 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.706139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.706991 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.706993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.707445 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.707446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.707584 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.708059 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:21.708066 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:21.710162 (+ 2096us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:21.712288 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.712289 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.712289 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.712511 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.712752 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.712752 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.712752 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.712810 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.713172 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:21.713173 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:21.713215 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 06:20:21.713287 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 06:20:21.714091 (+ 804us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"spinlock_wait_cycles":6912,"thread_start_us":90,"threads_started":1}
I20260926 06:20:21.714741 11986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.703694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.129:41703)
0926 06:20:21.704071 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:21.704081 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:21.704482 (+ 401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:21.704736 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:21.704740 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:21.705204 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:21.705270 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.705275 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.706241 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.706242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.707233 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.707235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.707631 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.707632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.707691 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.708199 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:21.708204 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:21.709672 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:21.711761 (+ 2089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.711767 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.711769 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.712028 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.712295 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.712296 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.712297 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.712434 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.712751 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:21.712759 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:21.712912 (+ 153us) client_negotiation.cc:769] Sending connection context
0926 06:20:21.713398 (+ 486us) client_negotiation.cc:240] Negotiation successful
0926 06:20:21.714537 (+ 1139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":1124,"thread_start_us":85,"threads_started":1}
I20260926 06:20:21.714919 11979 heartbeater.cc:347] Connected to a master server at 127.4.202.188:33663
I20260926 06:20:21.715001 11979 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:21.715070 11980 heartbeater.cc:347] Connected to a master server at 127.4.202.190:34455
I20260926 06:20:21.715132 11980 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:21.715181 11979 heartbeater.cc:511] Master 127.4.202.188:33663 requested a full tablet report, sending...
I20260926 06:20:21.715251 11980 heartbeater.cc:511] Master 127.4.202.190:34455 requested a full tablet report, sending...
I20260926 06:20:21.715621 11015 ts_manager.cc:194] Re-registered known tserver with Master: d47009f3689a4abfbbcd6623405bdcad (127.4.202.129:37589)
I20260926 06:20:21.715716 11800 ts_manager.cc:194] Registered new tserver with Master: d47009f3689a4abfbbcd6623405bdcad (127.4.202.129:37589)
I20260926 06:20:21.716147 11015 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.202.129@KRBTEST.COM'} at 127.4.202.129:41703
I20260926 06:20:21.719383 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.130:41657
--local_ip_for_outbound_sockets=127.4.202.130
--tserver_master_addrs=127.4.202.190:34455
--webserver_port=36517
--webserver_interface=127.4.202.130
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.130.keytab
--principal=oryx/127.4.202.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:46033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.4.202.190:34455,127.4.202.189:40149
--tserver_master_addrs=127.4.202.190:34455,127.4.202.189:40149,127.4.202.188:33663 with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:20:21.720131 11652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.704116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:47671 (local address 127.4.202.189:40149)
0926 06:20:21.704160 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.704162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.704693 (+ 531us) server_negotiation.cc:407] Connection header received
0926 06:20:21.704784 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.704786 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.704797 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.704845 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:21.705378 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.706146 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.707311 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.707578 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.707638 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.710855 (+ 3217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:21.710856 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:21.710857 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:21.710864 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:21.712544 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.719386 (+ 6842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.719387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.719387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.719438 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.719649 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.719650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.719650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.719758 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:21.719823 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:21.719977 (+ 154us) server_negotiation.cc:299] Negotiation successful
0926 06:20:21.720048 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 06:20:21.720315 11985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.703965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:40149 (local address 127.4.202.129:47671)
0926 06:20:21.704639 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:21.704649 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:21.704716 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:21.704877 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:21.704878 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:21.705166 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:21.705270 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.705275 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.706241 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.706242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.707233 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.707235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.707702 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.707702 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.707768 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.708361 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:21.708365 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:21.710760 (+ 2395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:21.719042 (+ 8282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.719043 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.719043 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.719308 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.719519 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.719520 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.719520 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.719584 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.719826 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:21.719827 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:21.719865 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 06:20:21.719941 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 06:20:21.720046 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"mutex_wait_us":29,"thread_start_us":65,"threads_started":1}
I20260926 06:20:21.720705 11981 heartbeater.cc:347] Connected to a master server at 127.4.202.189:40149
I20260926 06:20:21.720758 11981 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:21.720825 11981 heartbeater.cc:511] Master 127.4.202.189:40149 requested a full tablet report, sending...
I20260926 06:20:21.721172 11614 ts_manager.cc:194] Registered new tserver with Master: d47009f3689a4abfbbcd6623405bdcad (127.4.202.129:37589)
W20260926 06:20:21.798542 11987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:21.798777 11987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:21.798820 11987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:21.799923 11987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:21.799964 11987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:21.799988 11987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.130
I20260926 06:20:21.801294 11987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:46033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.130.keytab
--principal=oryx/127.4.202.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.130:41657
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.202.130
--webserver_port=36517
--tserver_master_addrs=127.4.202.190:34455,127.4.202.189:40149,127.4.202.188:33663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.130
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:21.801462 11987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:21.801614 11987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 06:20:21.806409 11987 server_base.cc:1034] running on GCE node
W20260926 06:20:21.806378 11993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:21.806420 11992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:21.806362 11995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:21.806895 11987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:21.807090 11987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:21.808260 11987 hybrid_clock.cc:648] HybridClock initialized: now 1790403621808249 us; error 39 us; skew 500 ppm
Sep 26 06:20:21 dist-test-slave-pd0t krb5kdc[10970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403621, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:21.810807 11987 init.cc:377] Logged in from keytab as oryx/127.4.202.130@KRBTEST.COM (short username oryx)
I20260926 06:20:21.811226 11987 webserver.cc:460] Webserver started at http://127.4.202.130:36517/ using document root <none> and password file <none>
I20260926 06:20:21.811468 11987 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:21.811551 11987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:21.812752 11987 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:20:21.813292 12002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:21.813465 11987 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:20:21.813565 11987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-1/wal
uuid: "20ac61fdecdf433ab60c8eb9b7f678cc"
format_stamp: "Formatted at 2026-09-26 06:20:20 on dist-test-slave-pd0t"
server_key: "f37d079f46c4ff890600f9a215087bb8"
server_key_iv: "0f1c96b1eaa0aff270744755b9164333"
server_key_version: "encryptionkey@0"
I20260926 06:20:21.813870 11987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:21.822008 11987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:21.823946 11987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:21.824092 11987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:21.824290 11987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:20:21.824559 11987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:20:21.824609 11987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:21.824648 11987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:20:21.824683 11987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:21.830355 11987 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.130:41657
I20260926 06:20:21.830444 12115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.130:41657 every 8 connection(s)
I20260926 06:20:21.830786 11987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
I20260926 06:20:21.834314 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 11987
I20260926 06:20:21.834468 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 11345
Sep 26 06:20:21 dist-test-slave-pd0t krb5kdc[10970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403621, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.188@KRBTEST.COM
Sep 26 06:20:21 dist-test-slave-pd0t krb5kdc[10970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403621, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.189@KRBTEST.COM
I20260926 06:20:21.839890 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.131:38777
--local_ip_for_outbound_sockets=127.4.202.131
--tserver_master_addrs=127.4.202.190:34455
--webserver_port=36607
--webserver_interface=127.4.202.131
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.131.keytab
--principal=oryx/127.4.202.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:46033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.4.202.190:34455,127.4.202.189:40149
--tserver_master_addrs=127.4.202.190:34455,127.4.202.189:40149,127.4.202.188:33663 with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:20:21.843057 11652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.832008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:43395 (local address 127.4.202.189:40149)
0926 06:20:21.832060 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.832063 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.832183 (+ 120us) server_negotiation.cc:407] Connection header received
0926 06:20:21.832921 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.832922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.832935 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.832968 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:21.833671 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.834395 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.836756 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.837034 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.837097 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.839803 (+ 2706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:21.839805 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:21.839805 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:21.839813 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:21.841265 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.841864 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.841865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.841865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.841923 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.842264 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.842265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.842265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.842411 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:21.842475 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:21.842840 (+ 365us) server_negotiation.cc:299] Negotiation successful
0926 06:20:21.842923 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
Sep 26 06:20:21 dist-test-slave-pd0t krb5kdc[10970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403621, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM
I20260926 06:20:21.843184 12122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.831768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:40149 (local address 127.4.202.130:43395)
0926 06:20:21.832112 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:21.832137 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:21.832811 (+ 674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:21.833035 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:21.833040 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:21.833434 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:21.833571 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.833577 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.834539 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.834540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.836234 (+ 1694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.836237 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.837809 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.837810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.837886 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.838320 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:21.838325 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:21.839702 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:21.841342 (+ 1640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.841348 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.841350 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.841768 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.842023 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.842024 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.842024 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.842195 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.842486 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:21.842492 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:21.842692 (+ 200us) client_negotiation.cc:769] Sending connection context
0926 06:20:21.842807 (+ 115us) client_negotiation.cc:240] Negotiation successful
0926 06:20:21.842962 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"mutex_wait_us":20,"thread_start_us":54,"threads_started":1}
I20260926 06:20:21.843654 12118 heartbeater.cc:347] Connected to a master server at 127.4.202.189:40149
I20260926 06:20:21.843745 12118 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:21.843922 12118 heartbeater.cc:511] Master 127.4.202.189:40149 requested a full tablet report, sending...
I20260926 06:20:21.844370 11614 ts_manager.cc:194] Registered new tserver with Master: 20ac61fdecdf433ab60c8eb9b7f678cc (127.4.202.130:41657)
I20260926 06:20:21.846613 11054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.831760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:32801 (local address 127.4.202.190:34455)
0926 06:20:21.831801 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.831803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.831813 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:20:21.832908 (+ 1095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.832909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.832923 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.832972 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:21.833658 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.834177 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.841213 (+ 7036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.841522 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.841568 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.844039 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:21.844040 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:21.844041 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:21.844048 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:21.845475 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.845809 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.845810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.845811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.845865 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.846059 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.846061 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.846061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.846176 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:21.846247 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:21.846433 (+ 186us) server_negotiation.cc:299] Negotiation successful
0926 06:20:21.846505 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:20:21.846638 12121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.831494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.130:32801)
0926 06:20:21.831731 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:21.831742 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:21.832802 (+ 1060us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:21.833034 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:21.833038 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:21.833373 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:21.833570 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.833574 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.834308 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.834309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.841106 (+ 6797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.841110 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.841649 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.841650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.841725 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.842160 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:21.842164 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:21.843900 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:21.845571 (+ 1671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.845572 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.845572 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.845740 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.845938 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.845939 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.845940 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.845994 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.846249 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:21.846250 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:21.846290 (+ 40us) client_negotiation.cc:769] Sending connection context
0926 06:20:21.846406 (+ 116us) client_negotiation.cc:240] Negotiation successful
0926 06:20:21.846501 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":95,"threads_started":1}
I20260926 06:20:21.847368 12117 heartbeater.cc:347] Connected to a master server at 127.4.202.190:34455
I20260926 06:20:21.847431 12117 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:21.847504 12117 heartbeater.cc:511] Master 127.4.202.190:34455 requested a full tablet report, sending...
I20260926 06:20:21.847738 11015 ts_manager.cc:194] Re-registered known tserver with Master: 20ac61fdecdf433ab60c8eb9b7f678cc (127.4.202.130:41657)
I20260926 06:20:21.848245 11015 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.202.130@KRBTEST.COM'} at 127.4.202.130:32801
I20260926 06:20:21.851137 12120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.831493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:33663 (local address 127.4.202.130:48989)
0926 06:20:21.831659 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:21.831696 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:21.832805 (+ 1109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:21.833035 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:21.833040 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:21.833421 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:21.833571 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.833577 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.834490 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.834491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.836234 (+ 1743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.836237 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.836702 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.836703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.836800 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.837292 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:21.837300 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:21.838842 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:21.841353 (+ 2511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.841354 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.841354 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.841769 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.842033 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.842034 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.842034 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.842180 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.850915 (+ 8735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:21.850915 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:21.850948 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 06:20:21.850996 (+ 48us) client_negotiation.cc:240] Negotiation successful
0926 06:20:21.851072 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":38,"thread_start_us":90,"threads_started":1}
I20260926 06:20:21.851227 11983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.831937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:48989 (local address 127.4.202.188:33663)
0926 06:20:21.831998 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.832000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.832008 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:20:21.832925 (+ 917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.832926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.832941 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.832972 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:21.833672 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.834404 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.836324 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.836617 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.836701 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.838949 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:21.838950 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:21.838951 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:21.838957 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:21.840643 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.841873 (+ 1230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.841874 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.841875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.841932 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.850707 (+ 8775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.850708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.850708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.850842 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:21.850917 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:21.851052 (+ 135us) server_negotiation.cc:299] Negotiation successful
0926 06:20:21.851138 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 06:20:21.851547 12116 heartbeater.cc:347] Connected to a master server at 127.4.202.188:33663
I20260926 06:20:21.851598 12116 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:21.851665 12116 heartbeater.cc:511] Master 127.4.202.188:33663 requested a full tablet report, sending...
I20260926 06:20:21.851914 11800 ts_manager.cc:194] Registered new tserver with Master: 20ac61fdecdf433ab60c8eb9b7f678cc (127.4.202.130:41657)
W20260926 06:20:21.921157 12123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:21.921336 12123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:21.921362 12123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:21.922552 12123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:21.922633 12123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:21.922663 12123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.131
I20260926 06:20:21.924242 12123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:46033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.131.keytab
--principal=oryx/127.4.202.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.131:38777
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.202.131
--webserver_port=36607
--tserver_master_addrs=127.4.202.190:34455,127.4.202.189:40149,127.4.202.188:33663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.131
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:21.924458 12123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:21.924718 12123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:21.929605 12128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:21.929596 12131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:21.929733 12129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:21.929836 12123 server_base.cc:1034] running on GCE node
I20260926 06:20:21.930027 12123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:21.930230 12123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:21.931361 12123 hybrid_clock.cc:648] HybridClock initialized: now 1790403621931339 us; error 36 us; skew 500 ppm
Sep 26 06:20:21 dist-test-slave-pd0t krb5kdc[10970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403621, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:21.933764 12123 init.cc:377] Logged in from keytab as oryx/127.4.202.131@KRBTEST.COM (short username oryx)
I20260926 06:20:21.934123 12123 webserver.cc:460] Webserver started at http://127.4.202.131:36607/ using document root <none> and password file <none>
I20260926 06:20:21.934300 12123 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:21.934351 12123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:21.935686 12123 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:20:21.936201 12138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:21.936383 12123 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:20:21.936484 12123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-2/wal
uuid: "a640f2f67fae4b4fb555f3ae032d02e4"
format_stamp: "Formatted at 2026-09-26 06:20:21 on dist-test-slave-pd0t"
server_key: "06a6e104136ba77923fabb901a466ceb"
server_key_iv: "79cdf5adcb46e15a03565fe1f226cb39"
server_key_version: "encryptionkey@0"
I20260926 06:20:21.936793 12123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:21.961963 12123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:21.963940 12123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:21.964128 12123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:21.964360 12123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:20:21.964651 12123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:20:21.964707 12123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:21.964748 12123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:20:21.964776 12123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:21.970259 12123 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.131:38777
I20260926 06:20:21.970300 12251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.131:38777 every 8 connection(s)
I20260926 06:20:21.970665 12123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
I20260926 06:20:21.975204 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 12123
Sep 26 06:20:21 dist-test-slave-pd0t krb5kdc[10970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403621, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM
Sep 26 06:20:21 dist-test-slave-pd0t krb5kdc[10970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403621, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.189@KRBTEST.COM
Sep 26 06:20:21 dist-test-slave-pd0t krb5kdc[10970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403621, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.188@KRBTEST.COM
I20260926 06:20:21.982822 11983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.971330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:48733 (local address 127.4.202.188:33663)
0926 06:20:21.971393 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.971396 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.971632 (+ 236us) server_negotiation.cc:407] Connection header received
0926 06:20:21.972725 (+ 1093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.972727 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.972741 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.972774 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:21.973496 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.974251 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.975397 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.975692 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.975733 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.979864 (+ 4131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:21.979865 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:21.979866 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:21.979873 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:21.981545 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.981922 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.981923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.981923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.981978 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.982165 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.982166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.982166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.982291 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:21.982364 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:21.982622 (+ 258us) server_negotiation.cc:299] Negotiation successful
0926 06:20:21.982708 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 06:20:21.982965 12258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.971637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:40149 (local address 127.4.202.131:33217)
0926 06:20:21.971779 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:21.971788 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:21.972608 (+ 820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:21.972844 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:21.972845 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:21.973258 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:21.973384 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.973390 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.974344 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.974345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.975302 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.975303 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.975810 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.975811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.975879 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.976534 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:21.976538 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:21.979004 (+ 2466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:21.980832 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.980838 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.980840 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.981131 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.981330 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.981330 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.981331 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.981452 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.982126 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:21.982133 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:21.982376 (+ 243us) client_negotiation.cc:769] Sending connection context
0926 06:20:21.982509 (+ 133us) client_negotiation.cc:240] Negotiation successful
0926 06:20:21.982723 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":90,"thread_start_us":61,"threads_started":1}
I20260926 06:20:21.983175 12257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.971340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:33663 (local address 127.4.202.131:48733)
0926 06:20:21.971521 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:21.971559 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:21.972608 (+ 1049us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:21.972844 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:21.972845 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:21.973263 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:21.973384 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.973390 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.974344 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.974345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.975296 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.975299 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.975811 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.975811 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.975879 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.976522 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:21.976531 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:21.979773 (+ 3242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:21.981635 (+ 1862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.981636 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.981636 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.981845 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.982045 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.982045 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.982046 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.982098 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.982392 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:21.982393 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:21.982437 (+ 44us) client_negotiation.cc:769] Sending connection context
0926 06:20:21.982519 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 06:20:21.983035 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":411,"thread_start_us":108,"threads_started":1}
I20260926 06:20:21.983413 11652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.971722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:33217 (local address 127.4.202.189:40149)
0926 06:20:21.971770 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.971773 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.971826 (+ 53us) server_negotiation.cc:407] Connection header received
0926 06:20:21.972728 (+ 902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.972729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.972740 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.972774 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:21.973488 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.974244 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.975397 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.975700 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.975735 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.979097 (+ 3362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:21.979098 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:21.979098 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:21.979105 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:21.980726 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.981213 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.981214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.981214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.981269 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.981522 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.981522 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.981522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.981637 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:21.982095 (+ 458us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:21.983091 (+ 996us) server_negotiation.cc:299] Negotiation successful
0926 06:20:21.983192 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 06:20:21.983759 12252 heartbeater.cc:347] Connected to a master server at 127.4.202.188:33663
I20260926 06:20:21.983857 12252 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:21.984022 12252 heartbeater.cc:511] Master 127.4.202.188:33663 requested a full tablet report, sending...
I20260926 06:20:21.984194 11054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.971338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:56147 (local address 127.4.202.190:34455)
0926 06:20:21.971397 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.971400 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.971682 (+ 282us) server_negotiation.cc:407] Connection header received
0926 06:20:21.972690 (+ 1008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.972692 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.972705 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.972760 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:21.973492 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.974011 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.975140 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.975459 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.975511 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.979536 (+ 4025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:21.979538 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:21.979538 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:21.979546 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:21.981893 (+ 2347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.982500 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.982501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.982502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.982561 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.982778 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.982779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.982779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.982911 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:21.982971 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:21.984032 (+ 1061us) server_negotiation.cc:299] Negotiation successful
0926 06:20:21.984104 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:20:21.984298 12254 heartbeater.cc:347] Connected to a master server at 127.4.202.189:40149
I20260926 06:20:21.984334 12254 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:21.984388 12254 heartbeater.cc:511] Master 127.4.202.189:40149 requested a full tablet report, sending...
I20260926 06:20:21.984638 11800 ts_manager.cc:194] Registered new tserver with Master: a640f2f67fae4b4fb555f3ae032d02e4 (127.4.202.131:38777)
I20260926 06:20:21.984722 11614 ts_manager.cc:194] Registered new tserver with Master: a640f2f67fae4b4fb555f3ae032d02e4 (127.4.202.131:38777)
I20260926 06:20:21.984933 12256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.971340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.131:56147)
0926 06:20:21.971526 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:21.971561 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:21.972572 (+ 1011us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:21.972825 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:21.972829 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:21.973195 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:21.973382 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.973387 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.974116 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.974117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.975029 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.975032 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.975599 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.975600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.975760 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.976612 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:21.976618 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:21.979448 (+ 2830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:21.981970 (+ 2522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.981971 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.981971 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.982167 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.982655 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:21.982656 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:21.982656 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:21.982712 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:21.983528 (+ 816us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:21.983529 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:21.983576 (+ 47us) client_negotiation.cc:769] Sending connection context
0926 06:20:21.983654 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 06:20:21.983746 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":2,"thread_start_us":125,"threads_started":1}
I20260926 06:20:21.985478 12253 heartbeater.cc:347] Connected to a master server at 127.4.202.190:34455
I20260926 06:20:21.985530 12253 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:21.985597 12253 heartbeater.cc:511] Master 127.4.202.190:34455 requested a full tablet report, sending...
I20260926 06:20:21.985832 11015 ts_manager.cc:194] Re-registered known tserver with Master: a640f2f67fae4b4fb555f3ae032d02e4 (127.4.202.131:38777)
I20260926 06:20:21.986778 11015 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.202.131@KRBTEST.COM'} at 127.4.202.131:56147
I20260926 06:20:21.989675 12265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.976438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36796 (local address 127.4.202.189:40149)
0926 06:20:21.977690 (+ 1252us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.977691 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.977699 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:20:21.977719 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.977720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.977731 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.977793 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:21.978558 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.979105 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.981125 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.981421 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.981478 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.986003 (+ 4525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:21.986004 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:21.986005 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:21.986012 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:21.988356 (+ 2344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.988754 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.988755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.988755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.988813 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.989015 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.989016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.989017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.989154 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:21.989249 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:21.989417 (+ 168us) server_negotiation.cc:299] Negotiation successful
0926 06:20:21.989494 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1220,"thread_start_us":46,"threads_started":1}
I20260926 06:20:21.989933 12263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.976298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39610 (local address 127.4.202.190:34455)
0926 06:20:21.977865 (+ 1567us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.977866 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.977872 (+ 6us) server_negotiation.cc:407] Connection header received
0926 06:20:21.977887 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.977887 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.977895 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.978009 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:21.979196 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.980013 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.984649 (+ 4636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.984927 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.984986 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.986819 (+ 1833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:21.986820 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:21.986821 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:21.986828 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:21.988567 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.988954 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.988955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.988955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.989020 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.989208 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.989210 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.989210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.989336 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:21.989438 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:21.989757 (+ 319us) server_negotiation.cc:299] Negotiation successful
0926 06:20:21.989847 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1539,"thread_start_us":94,"threads_started":1}
I20260926 06:20:21.990527 12268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.979234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36586 (local address 127.4.202.188:33663)
0926 06:20:21.980126 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.980127 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.980133 (+ 6us) server_negotiation.cc:407] Connection header received
0926 06:20:21.980151 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.980152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.980184 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.980727 (+ 543us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:21.983751 (+ 3024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.984540 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.985610 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.985885 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.985943 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.987445 (+ 1502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:21.987446 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:21.987447 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:21.987454 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:21.989207 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.989609 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.989610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.989610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.989667 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:21.990094 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:21.990095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:21.990095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:21.990208 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:21.990300 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:21.990412 (+ 112us) server_negotiation.cc:299] Negotiation successful
0926 06:20:21.990471 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":875,"thread_start_us":55,"threads_started":1}
I20260926 06:20:21.991513 11015 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39610:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:20:21.992970 11015 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:20:22.000200 12275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.996435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51066 (local address 127.4.202.129:37589)
0926 06:20:21.996595 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.996600 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.996612 (+ 12us) server_negotiation.cc:407] Connection header received
0926 06:20:21.996654 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.996656 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.996673 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.996925 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:20:21.996996 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.998094 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.999121 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.999843 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.999923 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:21.999998 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:22.000069 (+ 71us) server_negotiation.cc:299] Negotiation successful
0926 06:20:22.000105 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":81,"threads_started":1}
I20260926 06:20:22.000175 11573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.996219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:38777 (local address 127.0.0.1:37152)
0926 06:20:21.996250 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:21.996262 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:21.996301 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:21.996799 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:21.996799 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:21.996800 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:20:21.996851 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.996853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.998126 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.998127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.999153 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.999155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.999946 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.999948 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:22.000010 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:22.000014 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:20:22.000044 (+ 30us) client_negotiation.cc:240] Negotiation successful
0926 06:20:22.000084 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18,"mutex_wait_us":16}
I20260926 06:20:22.000243 12273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.996247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:37589 (local address 127.0.0.1:51066)
0926 06:20:21.996383 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:21.996394 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:21.996443 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:21.996789 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:21.996790 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:21.996792 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:20:21.996852 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.996853 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.998193 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.998194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.999036 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.999038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.999937 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.999938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:22.000004 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:22.000007 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:20:22.000060 (+ 53us) client_negotiation.cc:240] Negotiation successful
0926 06:20:22.000107 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":6,"thread_start_us":79,"threads_started":1}
I20260926 06:20:22.000432 12274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.996364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:41657 (local address 127.0.0.1:59782)
0926 06:20:21.996744 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:21.996753 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:21.996800 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:21.997030 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:21.997031 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:21.997033 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:20:21.997071 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.997073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.997954 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.997955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:21.999336 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:21.999337 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:22.000107 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:22.000108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:22.000166 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:22.000170 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:20:22.000216 (+ 46us) client_negotiation.cc:240] Negotiation successful
0926 06:20:22.000248 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":77,"threads_started":1}
I20260926 06:20:22.000574 12276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.996494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37152 (local address 127.4.202.131:38777)
0926 06:20:21.996636 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.996640 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.996648 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:20:21.996689 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.996691 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.996706 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.996912 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:20:21.996968 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.998047 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.999229 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.999853 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:22.000356 (+ 503us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:22.000443 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:22.000471 (+ 28us) server_negotiation.cc:299] Negotiation successful
0926 06:20:22.000511 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":70,"threads_started":1}
I20260926 06:20:22.000756 12277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:21.996522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59782 (local address 127.4.202.130:41657)
0926 06:20:21.996693 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:21.996697 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:21.996869 (+ 172us) server_negotiation.cc:407] Connection header received
0926 06:20:21.996911 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:21.996915 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:21.996933 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:21.997023 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:20:21.997139 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.997862 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:21.999423 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:21.999937 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:22.000004 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:22.000077 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:22.000486 (+ 409us) server_negotiation.cc:299] Negotiation successful
0926 06:20:22.000538 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":72,"threads_started":1}
I20260926 06:20:22.001073 11913 tablet_service.cc:1467] Processing CreateTablet for tablet 4f60d26a405b476cbf20d30d30ce333b (DEFAULT_TABLE table=test-table [id=19d39adfecfe442c92d7292c0ec0b04d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:20:22.001147 12186 tablet_service.cc:1467] Processing CreateTablet for tablet 4f60d26a405b476cbf20d30d30ce333b (DEFAULT_TABLE table=test-table [id=19d39adfecfe442c92d7292c0ec0b04d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:20:22.001302 12050 tablet_service.cc:1467] Processing CreateTablet for tablet 4f60d26a405b476cbf20d30d30ce333b (DEFAULT_TABLE table=test-table [id=19d39adfecfe442c92d7292c0ec0b04d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:20:22.001422 11913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f60d26a405b476cbf20d30d30ce333b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:22.001492 12186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f60d26a405b476cbf20d30d30ce333b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:22.001573 12050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f60d26a405b476cbf20d30d30ce333b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:22.004129 12278 tablet_bootstrap.cc:492] T 4f60d26a405b476cbf20d30d30ce333b P d47009f3689a4abfbbcd6623405bdcad: Bootstrap starting.
I20260926 06:20:22.004675 12278 tablet_bootstrap.cc:654] T 4f60d26a405b476cbf20d30d30ce333b P d47009f3689a4abfbbcd6623405bdcad: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:22.004910 12279 tablet_bootstrap.cc:492] T 4f60d26a405b476cbf20d30d30ce333b P 20ac61fdecdf433ab60c8eb9b7f678cc: Bootstrap starting.
I20260926 06:20:22.005029 12278 log.cc:824] T 4f60d26a405b476cbf20d30d30ce333b P d47009f3689a4abfbbcd6623405bdcad: Log is configured to *not* fsync() on all Append() calls
I20260926 06:20:22.005134 12280 tablet_bootstrap.cc:492] T 4f60d26a405b476cbf20d30d30ce333b P a640f2f67fae4b4fb555f3ae032d02e4: Bootstrap starting.
I20260926 06:20:22.006069 12279 tablet_bootstrap.cc:654] T 4f60d26a405b476cbf20d30d30ce333b P 20ac61fdecdf433ab60c8eb9b7f678cc: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:22.006148 12280 tablet_bootstrap.cc:654] T 4f60d26a405b476cbf20d30d30ce333b P a640f2f67fae4b4fb555f3ae032d02e4: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:22.006256 12278 tablet_bootstrap.cc:492] T 4f60d26a405b476cbf20d30d30ce333b P d47009f3689a4abfbbcd6623405bdcad: No bootstrap required, opened a new log
I20260926 06:20:22.006326 12278 ts_tablet_manager.cc:1397] T 4f60d26a405b476cbf20d30d30ce333b P d47009f3689a4abfbbcd6623405bdcad: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:20:22.006440 12279 log.cc:824] T 4f60d26a405b476cbf20d30d30ce333b P 20ac61fdecdf433ab60c8eb9b7f678cc: Log is configured to *not* fsync() on all Append() calls
I20260926 06:20:22.006470 12280 log.cc:824] T 4f60d26a405b476cbf20d30d30ce333b P a640f2f67fae4b4fb555f3ae032d02e4: Log is configured to *not* fsync() on all Append() calls
I20260926 06:20:22.007202 12279 tablet_bootstrap.cc:492] T 4f60d26a405b476cbf20d30d30ce333b P 20ac61fdecdf433ab60c8eb9b7f678cc: No bootstrap required, opened a new log
I20260926 06:20:22.007265 12279 ts_tablet_manager.cc:1397] T 4f60d26a405b476cbf20d30d30ce333b P 20ac61fdecdf433ab60c8eb9b7f678cc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:20:22.008747 12278 raft_consensus.cc:348] T 4f60d26a405b476cbf20d30d30ce333b P d47009f3689a4abfbbcd6623405bdcad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a640f2f67fae4b4fb555f3ae032d02e4" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 38777 } } peers { permanent_uuid: "d47009f3689a4abfbbcd6623405bdcad" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37589 } } peers { permanent_uuid: "20ac61fdecdf433ab60c8eb9b7f678cc" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 41657 } }
I20260926 06:20:22.008924 12278 raft_consensus.cc:374] T 4f60d26a405b476cbf20d30d30ce333b P d47009f3689a4abfbbcd6623405bdcad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:22.008966 12278 raft_consensus.cc:729] T 4f60d26a405b476cbf20d30d30ce333b P d47009f3689a4abfbbcd6623405bdcad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d47009f3689a4abfbbcd6623405bdcad, State: Initialized, Role: FOLLOWER
I20260926 06:20:22.009097 12278 consensus_queue.cc:260] T 4f60d26a405b476cbf20d30d30ce333b P d47009f3689a4abfbbcd6623405bdcad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a640f2f67fae4b4fb555f3ae032d02e4" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 38777 } } peers { permanent_uuid: "d47009f3689a4abfbbcd6623405bdcad" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37589 } } peers { permanent_uuid: "20ac61fdecdf433ab60c8eb9b7f678cc" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 41657 } }
I20260926 06:20:22.009380 12278 ts_tablet_manager.cc:1428] T 4f60d26a405b476cbf20d30d30ce333b P d47009f3689a4abfbbcd6623405bdcad: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 06:20:22.009502 11980 heartbeater.cc:503] Master 127.4.202.190:34455 was elected leader, sending a full tablet report...
I20260926 06:20:22.009778 12279 raft_consensus.cc:348] T 4f60d26a405b476cbf20d30d30ce333b P 20ac61fdecdf433ab60c8eb9b7f678cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a640f2f67fae4b4fb555f3ae032d02e4" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 38777 } } peers { permanent_uuid: "d47009f3689a4abfbbcd6623405bdcad" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37589 } } peers { permanent_uuid: "20ac61fdecdf433ab60c8eb9b7f678cc" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 41657 } }
I20260926 06:20:22.009936 12279 raft_consensus.cc:374] T 4f60d26a405b476cbf20d30d30ce333b P 20ac61fdecdf433ab60c8eb9b7f678cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:22.009977 12279 raft_consensus.cc:729] T 4f60d26a405b476cbf20d30d30ce333b P 20ac61fdecdf433ab60c8eb9b7f678cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20ac61fdecdf433ab60c8eb9b7f678cc, State: Initialized, Role: FOLLOWER
I20260926 06:20:22.010104 12279 consensus_queue.cc:260] T 4f60d26a405b476cbf20d30d30ce333b P 20ac61fdecdf433ab60c8eb9b7f678cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a640f2f67fae4b4fb555f3ae032d02e4" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 38777 } } peers { permanent_uuid: "d47009f3689a4abfbbcd6623405bdcad" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37589 } } peers { permanent_uuid: "20ac61fdecdf433ab60c8eb9b7f678cc" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 41657 } }
I20260926 06:20:22.010231 12280 tablet_bootstrap.cc:492] T 4f60d26a405b476cbf20d30d30ce333b P a640f2f67fae4b4fb555f3ae032d02e4: No bootstrap required, opened a new log
I20260926 06:20:22.010281 12280 ts_tablet_manager.cc:1397] T 4f60d26a405b476cbf20d30d30ce333b P a640f2f67fae4b4fb555f3ae032d02e4: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260926 06:20:22.010319 12279 ts_tablet_manager.cc:1428] T 4f60d26a405b476cbf20d30d30ce333b P 20ac61fdecdf433ab60c8eb9b7f678cc: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 06:20:22.010739 12117 heartbeater.cc:503] Master 127.4.202.190:34455 was elected leader, sending a full tablet report...
I20260926 06:20:22.012452 12280 raft_consensus.cc:348] T 4f60d26a405b476cbf20d30d30ce333b P a640f2f67fae4b4fb555f3ae032d02e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a640f2f67fae4b4fb555f3ae032d02e4" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 38777 } } peers { permanent_uuid: "d47009f3689a4abfbbcd6623405bdcad" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37589 } } peers { permanent_uuid: "20ac61fdecdf433ab60c8eb9b7f678cc" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 41657 } }
I20260926 06:20:22.012563 12280 raft_consensus.cc:374] T 4f60d26a405b476cbf20d30d30ce333b P a640f2f67fae4b4fb555f3ae032d02e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:22.012584 12280 raft_consensus.cc:729] T 4f60d26a405b476cbf20d30d30ce333b P a640f2f67fae4b4fb555f3ae032d02e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a640f2f67fae4b4fb555f3ae032d02e4, State: Initialized, Role: FOLLOWER
I20260926 06:20:22.012655 12280 consensus_queue.cc:260] T 4f60d26a405b476cbf20d30d30ce333b P a640f2f67fae4b4fb555f3ae032d02e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a640f2f67fae4b4fb555f3ae032d02e4" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 38777 } } peers { permanent_uuid: "d47009f3689a4abfbbcd6623405bdcad" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37589 } } peers { permanent_uuid: "20ac61fdecdf433ab60c8eb9b7f678cc" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 41657 } }
I20260926 06:20:22.012816 12280 ts_tablet_manager.cc:1428] T 4f60d26a405b476cbf20d30d30ce333b P a640f2f67fae4b4fb555f3ae032d02e4: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 06:20:22.012884 12253 heartbeater.cc:503] Master 127.4.202.190:34455 was elected leader, sending a full tablet report...
I20260926 06:20:22.049085 12286 raft_consensus.cc:482] T 4f60d26a405b476cbf20d30d30ce333b P a640f2f67fae4b4fb555f3ae032d02e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:20:22.049182 12286 raft_consensus.cc:504] T 4f60d26a405b476cbf20d30d30ce333b P a640f2f67fae4b4fb555f3ae032d02e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a640f2f67fae4b4fb555f3ae032d02e4" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 38777 } } peers { permanent_uuid: "d47009f3689a4abfbbcd6623405bdcad" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37589 } } peers { permanent_uuid: "20ac61fdecdf433ab60c8eb9b7f678cc" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 41657 } }
I20260926 06:20:22.049463 12286 leader_election.cc:288] T 4f60d26a405b476cbf20d30d30ce333b P a640f2f67fae4b4fb555f3ae032d02e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d47009f3689a4abfbbcd6623405bdcad (127.4.202.129:37589), 20ac61fdecdf433ab60c8eb9b7f678cc (127.4.202.130:41657)
I20260926 06:20:22.052443 12275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:22.049711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:32813 (local address 127.4.202.129:37589)
0926 06:20:22.049763 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:22.049765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:22.049774 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:20:22.049797 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:22.049798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:22.049806 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:22.049874 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:20:22.050049 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:22.050517 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:22.051550 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:22.052062 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:22.052191 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:22.052199 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:22.052346 (+ 147us) server_negotiation.cc:299] Negotiation successful
0926 06:20:22.052379 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 06:20:22.052474 12277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:22.049648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:33457 (local address 127.4.202.130:41657)
0926 06:20:22.049686 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:22.049688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:22.049720 (+ 32us) server_negotiation.cc:407] Connection header received
0926 06:20:22.049809 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:22.049810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:22.049816 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:22.049879 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:20:22.050087 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:22.050523 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:22.051614 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:22.052077 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:22.052188 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:22.052197 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:22.052363 (+ 166us) server_negotiation.cc:299] Negotiation successful
0926 06:20:22.052402 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 06:20:22.052479 12257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:22.049620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:41657 (local address 127.4.202.131:33457)
0926 06:20:22.049666 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:22.049677 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:22.049726 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:22.049885 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:22.049886 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:22.049888 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:20:22.049966 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:22.049967 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:22.050636 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:22.050637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:22.051497 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:22.051499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:22.052224 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:22.052225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:22.052292 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:22.052294 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:20:22.052348 (+ 54us) client_negotiation.cc:240] Negotiation successful
0926 06:20:22.052383 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":4}
I20260926 06:20:22.052487 12256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:22.049597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:37589 (local address 127.4.202.131:32813)
0926 06:20:22.049665 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:22.049677 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:22.049719 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:22.049891 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:22.049892 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:22.049893 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:20:22.049962 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:22.049963 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:22.050623 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:22.050624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:22.051470 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:22.051472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:22.052191 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:22.052192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:22.052267 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:22.052270 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:20:22.052335 (+ 65us) client_negotiation.cc:240] Negotiation successful
0926 06:20:22.052383 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":47}
I20260926 06:20:22.052795 12070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f60d26a405b476cbf20d30d30ce333b" candidate_uuid: "a640f2f67fae4b4fb555f3ae032d02e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20ac61fdecdf433ab60c8eb9b7f678cc" is_pre_election: true
I20260926 06:20:22.052935 12070 raft_consensus.cc:2454] T 4f60d26a405b476cbf20d30d30ce333b P 20ac61fdecdf433ab60c8eb9b7f678cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a640f2f67fae4b4fb555f3ae032d02e4 in term 0.
I20260926 06:20:22.053158 12142 leader_election.cc:302] T 4f60d26a405b476cbf20d30d30ce333b P a640f2f67fae4b4fb555f3ae032d02e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20ac61fdecdf433ab60c8eb9b7f678cc, a640f2f67fae4b4fb555f3ae032d02e4; no voters:
I20260926 06:20:22.053165 11933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f60d26a405b476cbf20d30d30ce333b" candidate_uuid: "a640f2f67fae4b4fb555f3ae032d02e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d47009f3689a4abfbbcd6623405bdcad" is_pre_election: true
I20260926 06:20:22.053287 11933 raft_consensus.cc:2454] T 4f60d26a405b476cbf20d30d30ce333b P d47009f3689a4abfbbcd6623405bdcad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a640f2f67fae4b4fb555f3ae032d02e4 in term 0.
I20260926 06:20:22.053308 12286 raft_consensus.cc:2781] T 4f60d26a405b476cbf20d30d30ce333b P a640f2f67fae4b4fb555f3ae032d02e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:20:22.053376 12286 raft_consensus.cc:482] T 4f60d26a405b476cbf20d30d30ce333b P a640f2f67fae4b4fb555f3ae032d02e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:20:22.053412 12286 raft_consensus.cc:3037] T 4f60d26a405b476cbf20d30d30ce333b P a640f2f67fae4b4fb555f3ae032d02e4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:22.054055 12286 raft_consensus.cc:504] T 4f60d26a405b476cbf20d30d30ce333b P a640f2f67fae4b4fb555f3ae032d02e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a640f2f67fae4b4fb555f3ae032d02e4" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 38777 } } peers { permanent_uuid: "d47009f3689a4abfbbcd6623405bdcad" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37589 } } peers { permanent_uuid: "20ac61fdecdf433ab60c8eb9b7f678cc" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 41657 } }
I20260926 06:20:22.054203 12286 leader_election.cc:288] T 4f60d26a405b476cbf20d30d30ce333b P a640f2f67fae4b4fb555f3ae032d02e4 [CANDIDATE]: Term 1 election: Requested vote from peers d47009f3689a4abfbbcd6623405bdcad (127.4.202.129:37589), 20ac61fdecdf433ab60c8eb9b7f678cc (127.4.202.130:41657)
I20260926 06:20:22.054339 12070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f60d26a405b476cbf20d30d30ce333b" candidate_uuid: "a640f2f67fae4b4fb555f3ae032d02e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20ac61fdecdf433ab60c8eb9b7f678cc"
I20260926 06:20:22.054376 11933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f60d26a405b476cbf20d30d30ce333b" candidate_uuid: "a640f2f67fae4b4fb555f3ae032d02e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d47009f3689a4abfbbcd6623405bdcad"
I20260926 06:20:22.054436 12070 raft_consensus.cc:3037] T 4f60d26a405b476cbf20d30d30ce333b P 20ac61fdecdf433ab60c8eb9b7f678cc [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:22.054435 11933 raft_consensus.cc:3037] T 4f60d26a405b476cbf20d30d30ce333b P d47009f3689a4abfbbcd6623405bdcad [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:22.055234 12070 raft_consensus.cc:2454] T 4f60d26a405b476cbf20d30d30ce333b P 20ac61fdecdf433ab60c8eb9b7f678cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a640f2f67fae4b4fb555f3ae032d02e4 in term 1.
I20260926 06:20:22.055433 12142 leader_election.cc:302] T 4f60d26a405b476cbf20d30d30ce333b P a640f2f67fae4b4fb555f3ae032d02e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20ac61fdecdf433ab60c8eb9b7f678cc, a640f2f67fae4b4fb555f3ae032d02e4; no voters:
I20260926 06:20:22.055563 12286 raft_consensus.cc:2781] T 4f60d26a405b476cbf20d30d30ce333b P a640f2f67fae4b4fb555f3ae032d02e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:20:22.055723 12286 raft_consensus.cc:686] T 4f60d26a405b476cbf20d30d30ce333b P a640f2f67fae4b4fb555f3ae032d02e4 [term 1 LEADER]: Becoming Leader. State: Replica: a640f2f67fae4b4fb555f3ae032d02e4, State: Running, Role: LEADER
I20260926 06:20:22.055786 11933 raft_consensus.cc:2454] T 4f60d26a405b476cbf20d30d30ce333b P d47009f3689a4abfbbcd6623405bdcad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a640f2f67fae4b4fb555f3ae032d02e4 in term 1.
I20260926 06:20:22.055827 12286 consensus_queue.cc:237] T 4f60d26a405b476cbf20d30d30ce333b P a640f2f67fae4b4fb555f3ae032d02e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a640f2f67fae4b4fb555f3ae032d02e4" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 38777 } } peers { permanent_uuid: "d47009f3689a4abfbbcd6623405bdcad" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37589 } } peers { permanent_uuid: "20ac61fdecdf433ab60c8eb9b7f678cc" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 41657 } }
I20260926 06:20:22.056768 11014 catalog_manager.cc:5495] T 4f60d26a405b476cbf20d30d30ce333b P a640f2f67fae4b4fb555f3ae032d02e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to a640f2f67fae4b4fb555f3ae032d02e4 (127.4.202.131). New cstate: current_term: 1 leader_uuid: "a640f2f67fae4b4fb555f3ae032d02e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a640f2f67fae4b4fb555f3ae032d02e4" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 38777 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d47009f3689a4abfbbcd6623405bdcad" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20ac61fdecdf433ab60c8eb9b7f678cc" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 41657 } health_report { overall_health: UNKNOWN } } }
I20260926 06:20:22.075507 12276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:22.073023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37160 (local address 127.4.202.131:38777)
0926 06:20:22.073078 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:22.073080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:22.073089 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:20:22.073177 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:22.073178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:22.073195 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:22.073277 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:20:22.073411 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:22.073884 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:22.074693 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:22.074918 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:22.074993 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:22.075171 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:20:22.075290 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:20:22.075382 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:22.075408 (+ 26us) server_negotiation.cc:299] Negotiation successful
0926 06:20:22.075428 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 06:20:22.077193 12070 raft_consensus.cc:1261] T 4f60d26a405b476cbf20d30d30ce333b P 20ac61fdecdf433ab60c8eb9b7f678cc [term 1 FOLLOWER]: Refusing update from remote peer a640f2f67fae4b4fb555f3ae032d02e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:20:22.077242 11933 raft_consensus.cc:1261] T 4f60d26a405b476cbf20d30d30ce333b P d47009f3689a4abfbbcd6623405bdcad [term 1 FOLLOWER]: Refusing update from remote peer a640f2f67fae4b4fb555f3ae032d02e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:20:22.077533 12286 consensus_queue.cc:1035] T 4f60d26a405b476cbf20d30d30ce333b P a640f2f67fae4b4fb555f3ae032d02e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20ac61fdecdf433ab60c8eb9b7f678cc" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 41657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:20:22.077696 12287 consensus_queue.cc:1035] T 4f60d26a405b476cbf20d30d30ce333b P a640f2f67fae4b4fb555f3ae032d02e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d47009f3689a4abfbbcd6623405bdcad" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:20:22.080713 12291 mvcc.cc:205] Tried to move back new op lower bound from 7333493236025712640 to 7333493235941195776. Current Snapshot: MvccSnapshot[applied={T|T < 7333493236025712640}]
W20260926 06:20:22.081458 12119 tablet.cc:2369] T 4f60d26a405b476cbf20d30d30ce333b P 20ac61fdecdf433ab60c8eb9b7f678cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:20:22.083027 11015 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:20:22.083108 11015 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:20:22.083380 12292 mvcc.cc:205] Tried to move back new op lower bound from 7333493236025712640 to 7333493235941195776. Current Snapshot: MvccSnapshot[applied={T|T < 7333493236025712640}]
I20260926 06:20:22.084653 11015 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7490610226774824aaae08390ef4c1e1: Sending DeleteTablet for 3 replicas of tablet 4f60d26a405b476cbf20d30d30ce333b
I20260926 06:20:22.084936 12274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:22.080684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:33663 (local address 127.0.0.1:36596)
0926 06:20:22.080815 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:22.080826 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:22.080868 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:22.081648 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:22.081649 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:22.081651 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:20:22.081706 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:22.081708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:22.082561 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:22.082562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:22.083628 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:22.083630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:22.084594 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:22.084595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:22.084660 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:22.084664 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:20:22.084732 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 06:20:22.084776 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114}
I20260926 06:20:22.085110 11913 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f60d26a405b476cbf20d30d30ce333b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:20:22 UTC) from {username='oryx', principal='oryx/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:51066
I20260926 06:20:22.085134 12050 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f60d26a405b476cbf20d30d30ce333b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:20:22 UTC) from {username='oryx', principal='oryx/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:59782
I20260926 06:20:22.085249 12268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:22.080966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36596 (local address 127.4.202.188:33663)
0926 06:20:22.081025 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:22.081027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:22.081035 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:20:22.081057 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:22.081058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:22.081080 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:22.081136 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:20:22.081768 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:22.082460 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:22.083735 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:22.084306 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:22.084375 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:22.084448 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:22.085010 (+ 562us) server_negotiation.cc:299] Negotiation successful
0926 06:20:22.085129 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 06:20:22.085304 12049 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f60d26a405b476cbf20d30d30ce333b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:20:22 UTC) from {username='oryx', principal='oryx/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:59782
I20260926 06:20:22.085427 12300 tablet_replica.cc:330] T 4f60d26a405b476cbf20d30d30ce333b P d47009f3689a4abfbbcd6623405bdcad: stopping tablet replica
I20260926 06:20:22.085517 12300 raft_consensus.cc:2229] T 4f60d26a405b476cbf20d30d30ce333b P d47009f3689a4abfbbcd6623405bdcad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:20:22.085608 12300 raft_consensus.cc:2258] T 4f60d26a405b476cbf20d30d30ce333b P d47009f3689a4abfbbcd6623405bdcad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:20:22.085758 12299 tablet_replica.cc:330] T 4f60d26a405b476cbf20d30d30ce333b P 20ac61fdecdf433ab60c8eb9b7f678cc: stopping tablet replica
I20260926 06:20:22.085875 12299 raft_consensus.cc:2229] T 4f60d26a405b476cbf20d30d30ce333b P 20ac61fdecdf433ab60c8eb9b7f678cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:20:22.085959 12299 raft_consensus.cc:2258] T 4f60d26a405b476cbf20d30d30ce333b P 20ac61fdecdf433ab60c8eb9b7f678cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:20:22.085913 11810 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.10] Dedup: 1.7->[1.8-1.10]
I20260926 06:20:22.086200 12186 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f60d26a405b476cbf20d30d30ce333b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:20:22 UTC) from {username='oryx', principal='oryx/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:37152
I20260926 06:20:22.086339 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 10983
I20260926 06:20:22.086400 12302 tablet_replica.cc:330] T 4f60d26a405b476cbf20d30d30ce333b P a640f2f67fae4b4fb555f3ae032d02e4: stopping tablet replica
I20260926 06:20:22.086912 12302 raft_consensus.cc:2229] T 4f60d26a405b476cbf20d30d30ce333b P a640f2f67fae4b4fb555f3ae032d02e4 [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:20:22.087008 12302 raft_consensus.cc:2258] T 4f60d26a405b476cbf20d30d30ce333b P a640f2f67fae4b4fb555f3ae032d02e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:20:22.087298 11839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7490610226774824aaae08390ef4c1e1. Latest consensus state: current_term: 1 leader_uuid: "7490610226774824aaae08390ef4c1e1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 40149 } attrs { promote: false } } peers { permanent_uuid: "092902ed31314ba181f705bd4701e4c7" member_type: NON_VOTER last_known_addr { host: "127.4.202.188" port: 33663 } attrs { promote: true } } }
I20260926 06:20:22.087384 11839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 092902ed31314ba181f705bd4701e4c7 [sys.catalog]: This master's current role is: LEARNER
I20260926 06:20:22.087680 12299 ts_tablet_manager.cc:1905] T 4f60d26a405b476cbf20d30d30ce333b P 20ac61fdecdf433ab60c8eb9b7f678cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:20:22.088053 12300 ts_tablet_manager.cc:1905] T 4f60d26a405b476cbf20d30d30ce333b P d47009f3689a4abfbbcd6623405bdcad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:20:22.089668 12299 ts_tablet_manager.cc:1918] T 4f60d26a405b476cbf20d30d30ce333b P 20ac61fdecdf433ab60c8eb9b7f678cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:20:22.089748 12299 log.cc:1196] T 4f60d26a405b476cbf20d30d30ce333b P 20ac61fdecdf433ab60c8eb9b7f678cc: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-1/wal/wals/4f60d26a405b476cbf20d30d30ce333b
I20260926 06:20:22.089782 12300 ts_tablet_manager.cc:1918] T 4f60d26a405b476cbf20d30d30ce333b P d47009f3689a4abfbbcd6623405bdcad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:20:22.089859 12300 log.cc:1196] T 4f60d26a405b476cbf20d30d30ce333b P d47009f3689a4abfbbcd6623405bdcad: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-0/wal/wals/4f60d26a405b476cbf20d30d30ce333b
I20260926 06:20:22.090004 12299 ts_tablet_manager.cc:1939] T 4f60d26a405b476cbf20d30d30ce333b P 20ac61fdecdf433ab60c8eb9b7f678cc: Deleting consensus metadata
I20260926 06:20:22.090176 12300 ts_tablet_manager.cc:1939] T 4f60d26a405b476cbf20d30d30ce333b P d47009f3689a4abfbbcd6623405bdcad: Deleting consensus metadata
I20260926 06:20:22.090698 12302 ts_tablet_manager.cc:1905] T 4f60d26a405b476cbf20d30d30ce333b P a640f2f67fae4b4fb555f3ae032d02e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:20:22.092651 12302 ts_tablet_manager.cc:1918] T 4f60d26a405b476cbf20d30d30ce333b P a640f2f67fae4b4fb555f3ae032d02e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:20:22.092725 12302 log.cc:1196] T 4f60d26a405b476cbf20d30d30ce333b P a640f2f67fae4b4fb555f3ae032d02e4: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0/minicluster-data/ts-2/wal/wals/4f60d26a405b476cbf20d30d30ce333b
I20260926 06:20:22.092957 12302 ts_tablet_manager.cc:1939] T 4f60d26a405b476cbf20d30d30ce333b P a640f2f67fae4b4fb555f3ae032d02e4: Deleting consensus metadata
W20260926 06:20:22.093479 11786 connection.cc:535] server connection from 127.0.0.1:36596 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 06:20:22.093509 12003 connection.cc:535] server connection from 127.0.0.1:59782 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 06:20:22.093797 11867 connection.cc:535] server connection from 127.0.0.1:51066 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 06:20:22.094067 12003 connection.cc:441] server connection from 127.0.0.1:59782 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59782 (request call id 2) could send its response
W20260926 06:20:22.094909 12139 connection.cc:535] server connection from 127.0.0.1:37152 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 06:20:22.101667 12268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:22.086969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36612 (local address 127.4.202.188:33663)
0926 06:20:22.087148 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:22.087150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:22.087159 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:20:22.087202 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:22.087203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:22.087213 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:22.087282 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:22.093958 (+ 6676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:22.094760 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:22.096344 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:22.096651 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:22.096731 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:22.098046 (+ 1315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:22.098047 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:22.098047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:22.098054 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:22.100063 (+ 2009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:22.100449 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:22.100451 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:22.100451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:22.100783 (+ 332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:22.101039 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:22.101040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:22.101040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:22.101185 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:22.101264 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:22.101500 (+ 236us) server_negotiation.cc:299] Negotiation successful
0926 06:20:22.101576 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160}
I20260926 06:20:22.103369 12265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:22.095823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36808 (local address 127.4.202.189:40149)
0926 06:20:22.095874 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:22.095876 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:22.095883 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:20:22.095903 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:22.095904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:22.095915 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:22.095959 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:22.096632 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:22.097321 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:22.098407 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:22.098774 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:22.098844 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:22.100225 (+ 1381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:22.100227 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:22.100227 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:22.100234 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:22.101948 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:22.102615 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:22.102616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:22.102617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:22.102672 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:22.102853 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:22.102854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:22.102854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:22.102979 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:22.103083 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:22.103175 (+ 92us) server_negotiation.cc:299] Negotiation successful
0926 06:20:22.103253 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260926 06:20:22.105600 11983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:22.095735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36626 (local address 127.4.202.188:33663)
0926 06:20:22.095773 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:22.095776 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:22.098724 (+ 2948us) server_negotiation.cc:407] Connection header received
0926 06:20:22.098820 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:22.098821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:22.098830 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:22.098859 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:22.099471 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:22.100223 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:22.101267 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:22.101567 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:22.101603 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:22.103269 (+ 1666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:22.103271 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:22.103271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:22.103277 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:22.104667 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:22.105020 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:22.105020 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:22.105021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:22.105065 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:22.105239 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:22.105239 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:22.105240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:22.105315 (+ 75us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:22.105378 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:22.105503 (+ 125us) server_negotiation.cc:299] Negotiation successful
0926 06:20:22.105550 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 06:20:23.082783 12313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:23.082484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.131:49121)
0926 06:20:23.082637 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:23.082726 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":64,"threads_started":1}
I20260926 06:20:23.083011 12314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:23.082654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.129:43119)
0926 06:20:23.082770 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:23.082956 (+ 186us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":58,"threads_started":1}
W20260926 06:20:23.083036 12253 heartbeater.cc:644] Failed to heartbeat to 127.4.202.190:34455 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
W20260926 06:20:23.083237 11980 heartbeater.cc:644] Failed to heartbeat to 127.4.202.190:34455 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
I20260926 06:20:23.085347 12315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:23.085146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.130:36297)
0926 06:20:23.085257 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:23.085313 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":44,"threads_started":1}
W20260926 06:20:23.085553 12117 heartbeater.cc:644] Failed to heartbeat to 127.4.202.190:34455 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
I20260926 06:20:23.585393 12316 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7490610226774824aaae08390ef4c1e1)
I20260926 06:20:23.585541 12316 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 40149 } attrs { promote: false } } peers { permanent_uuid: "092902ed31314ba181f705bd4701e4c7" member_type: NON_VOTER last_known_addr { host: "127.4.202.188" port: 33663 } attrs { promote: true } }
I20260926 06:20:23.585764 12316 leader_election.cc:288] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7490610226774824aaae08390ef4c1e1 (127.4.202.190:34455)
I20260926 06:20:23.586068 12317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:23.585886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.0.0.1:40078)
0926 06:20:23.585979 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:23.586025 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":55,"threads_started":1}
W20260926 06:20:23.586195 11600 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
I20260926 06:20:23.586544 12317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:23.586453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.0.0.1:40088)
0926 06:20:23.586502 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:23.586528 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260926 06:20:23.586673 11600 leader_election.cc:334] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7490610226774824aaae08390ef4c1e1 (127.4.202.190:34455): Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
I20260926 06:20:23.586753 11600 leader_election.cc:302] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 1245bf7b57c549d28ef3f35463029b01; no voters: 7490610226774824aaae08390ef4c1e1
I20260926 06:20:23.586848 12316 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 06:20:24.083598 12319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:24.083367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.131:38803)
0926 06:20:24.083497 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:24.083539 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":57,"threads_started":1}
I20260926 06:20:24.083757 12320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:24.083535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.129:48979)
0926 06:20:24.083634 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:24.083673 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":55,"threads_started":1}
I20260926 06:20:24.085960 12321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:24.085824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.130:42931)
0926 06:20:24.085903 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:24.085926 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69,"thread_start_us":39,"threads_started":1}
I20260926 06:20:25.084275 12323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:25.084086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.129:38751)
0926 06:20:25.084173 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:25.084241 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66,"thread_start_us":38,"threads_started":1}
I20260926 06:20:25.084270 12322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:25.084028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.131:47393)
0926 06:20:25.084185 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:25.084218 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":58,"threads_started":1}
I20260926 06:20:25.086539 12324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:25.086355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.130:54013)
0926 06:20:25.086467 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:25.086500 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":69,"threads_started":1}
I20260926 06:20:25.721391 12325 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7490610226774824aaae08390ef4c1e1)
I20260926 06:20:25.721511 12325 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 40149 } attrs { promote: false } } peers { permanent_uuid: "092902ed31314ba181f705bd4701e4c7" member_type: NON_VOTER last_known_addr { host: "127.4.202.188" port: 33663 } attrs { promote: true } }
I20260926 06:20:25.721683 12325 leader_election.cc:288] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7490610226774824aaae08390ef4c1e1 (127.4.202.190:34455)
I20260926 06:20:25.722016 12326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:25.721844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.0.0.1:50432)
0926 06:20:25.721940 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:25.721983 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78,"thread_start_us":57,"threads_started":1}
I20260926 06:20:25.722445 12326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:25.722318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.0.0.1:50444)
0926 06:20:25.722367 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:25.722417 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260926 06:20:25.722503 11600 leader_election.cc:334] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7490610226774824aaae08390ef4c1e1 (127.4.202.190:34455): Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
I20260926 06:20:25.722534 11600 leader_election.cc:302] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 1245bf7b57c549d28ef3f35463029b01; no voters: 7490610226774824aaae08390ef4c1e1
I20260926 06:20:25.722632 12325 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 06:20:26.084988 12329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:26.084773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.129:36035)
0926 06:20:26.084893 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:26.084932 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":58,"threads_started":1}
I20260926 06:20:26.085021 12328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:26.084726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.131:57373)
0926 06:20:26.084941 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:26.084976 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":92,"threads_started":1}
I20260926 06:20:26.087182 12330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:26.086998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.130:39101)
0926 06:20:26.087114 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:26.087148 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":45,"threads_started":1}
I20260926 06:20:27.085669 12331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:27.085390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.131:36569)
0926 06:20:27.085559 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:27.085610 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":57,"threads_started":1}
I20260926 06:20:27.085829 12332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:27.085567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.129:37387)
0926 06:20:27.085736 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:27.085771 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":44,"threads_started":1}
I20260926 06:20:27.087744 12333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:27.087568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.130:50647)
0926 06:20:27.087679 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:27.087711 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":57,"threads_started":1}
I20260926 06:20:27.428167 12334 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7490610226774824aaae08390ef4c1e1)
I20260926 06:20:27.428268 12334 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 40149 } attrs { promote: false } } peers { permanent_uuid: "092902ed31314ba181f705bd4701e4c7" member_type: NON_VOTER last_known_addr { host: "127.4.202.188" port: 33663 } attrs { promote: true } }
I20260926 06:20:27.428474 12334 leader_election.cc:288] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7490610226774824aaae08390ef4c1e1 (127.4.202.190:34455)
I20260926 06:20:27.428828 12335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:27.428644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.0.0.1:50594)
0926 06:20:27.428749 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:27.428791 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":69,"threads_started":1}
I20260926 06:20:27.429309 12335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:27.429216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.0.0.1:50606)
0926 06:20:27.429265 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:27.429291 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260926 06:20:27.429421 11600 leader_election.cc:334] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7490610226774824aaae08390ef4c1e1 (127.4.202.190:34455): Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
I20260926 06:20:27.429486 11600 leader_election.cc:302] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 1245bf7b57c549d28ef3f35463029b01; no voters: 7490610226774824aaae08390ef4c1e1
I20260926 06:20:27.429554 12334 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 06:20:28.086526 12338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:28.086236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.129:43483)
0926 06:20:28.086431 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:28.086472 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":52,"threads_started":1}
I20260926 06:20:28.086525 12337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:28.086134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.131:40993)
0926 06:20:28.086431 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:28.086474 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":58,"threads_started":1}
I20260926 06:20:28.088296 12339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:28.088116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.130:50295)
0926 06:20:28.088228 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:28.088263 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":47,"threads_started":1}
I20260926 06:20:29.087437 12341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:29.087064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.129:46227)
0926 06:20:29.087334 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:29.087382 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":63,"threads_started":1}
I20260926 06:20:29.087591 12340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:29.087064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.131:37365)
0926 06:20:29.087503 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:29.087546 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":77,"threads_started":1}
I20260926 06:20:29.088871 12342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:29.088698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.130:45017)
0926 06:20:29.088803 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:29.088837 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":51,"threads_started":1}
I20260926 06:20:30.088141 12343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:30.087919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.129:48151)
0926 06:20:30.088047 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:30.088090 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":61,"threads_started":1}
I20260926 06:20:30.088366 12344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:30.088041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.131:37491)
0926 06:20:30.088278 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:30.088319 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":56,"threads_started":1}
I20260926 06:20:30.089448 12345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:30.089307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.130:56761)
0926 06:20:30.089393 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:30.089416 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74,"thread_start_us":41,"threads_started":1}
I20260926 06:20:31.089063 12346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:31.088652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.129:39121)
0926 06:20:31.088963 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:31.089009 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":95,"threads_started":1}
I20260926 06:20:31.089123 12347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:31.088913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.131:39223)
0926 06:20:31.089051 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:31.089083 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":51,"threads_started":1}
I20260926 06:20:31.089978 12348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:31.089814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.130:39547)
0926 06:20:31.089909 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:31.089946 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79,"thread_start_us":59,"threads_started":1}
I20260926 06:20:32.048283 12349 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7490610226774824aaae08390ef4c1e1)
I20260926 06:20:32.048374 12349 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 40149 } attrs { promote: false } } peers { permanent_uuid: "092902ed31314ba181f705bd4701e4c7" member_type: NON_VOTER last_known_addr { host: "127.4.202.188" port: 33663 } attrs { promote: true } }
I20260926 06:20:32.048561 12349 leader_election.cc:288] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7490610226774824aaae08390ef4c1e1 (127.4.202.190:34455)
I20260926 06:20:32.048872 12350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:32.048687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.0.0.1:50980)
0926 06:20:32.048793 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:32.048836 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":65,"threads_started":1}
W20260926 06:20:32.048951 11600 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111) [suppressed 2 similar messages]
I20260926 06:20:32.049341 12350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:32.049210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.0.0.1:50988)
0926 06:20:32.049295 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:32.049322 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260926 06:20:32.049422 11600 leader_election.cc:334] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7490610226774824aaae08390ef4c1e1 (127.4.202.190:34455): Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
I20260926 06:20:32.049463 11600 leader_election.cc:302] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 1245bf7b57c549d28ef3f35463029b01; no voters: 7490610226774824aaae08390ef4c1e1
I20260926 06:20:32.049525 12349 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 06:20:32.089999 12353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:32.089665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.129:40917)
0926 06:20:32.089908 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:32.089950 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":78,"threads_started":1}
I20260926 06:20:32.090003 12352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:32.089642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.131:55295)
0926 06:20:32.089914 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:32.089951 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":58,"threads_started":1}
I20260926 06:20:32.090507 12354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:32.090334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.130:40763)
0926 06:20:32.090441 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:32.090474 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":51,"threads_started":1}
I20260926 06:20:33.090581 12355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:33.090353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.131:53245)
0926 06:20:33.090485 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:33.090523 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":75,"threads_started":1}
I20260926 06:20:33.090693 12356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:33.090360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.129:35769)
0926 06:20:33.090607 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:33.090645 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":69,"threads_started":1}
I20260926 06:20:33.091015 12357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:33.090863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.130:39871)
0926 06:20:33.090962 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:33.090985 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":45,"threads_started":1}
I20260926 06:20:34.091565 12358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:34.091089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.129:39513)
0926 06:20:34.091465 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:34.091507 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":66,"threads_started":1}
I20260926 06:20:34.091603 12359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:34.091130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.131:44729)
0926 06:20:34.091471 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:34.091551 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":42,"threads_started":1}
I20260926 06:20:34.091655 12360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:34.091351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.130:38231)
0926 06:20:34.091587 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:34.091618 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":46,"threads_started":1}
I20260926 06:20:35.092309 12362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:35.092092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.131:59807)
0926 06:20:35.092221 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:35.092257 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":52,"threads_started":1}
I20260926 06:20:35.092319 12363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:35.092101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.129:37691)
0926 06:20:35.092221 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:35.092282 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":47,"threads_started":1}
I20260926 06:20:35.092371 12361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:35.092022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.130:43859)
0926 06:20:35.092280 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:35.092319 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":59,"threads_started":1}
I20260926 06:20:36.093055 12365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:36.092768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.131:49701)
0926 06:20:36.092956 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:36.092996 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":54,"threads_started":1}
I20260926 06:20:36.093096 12364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:36.092747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.129:34565)
0926 06:20:36.093027 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:36.093055 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":64,"threads_started":1}
I20260926 06:20:36.093312 12366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:36.092987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.130:37455)
0926 06:20:36.093216 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:36.093265 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":58,"threads_started":1}
I20260926 06:20:36.809959 12367 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7490610226774824aaae08390ef4c1e1)
I20260926 06:20:36.810074 12367 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 40149 } attrs { promote: false } } peers { permanent_uuid: "092902ed31314ba181f705bd4701e4c7" member_type: NON_VOTER last_known_addr { host: "127.4.202.188" port: 33663 } attrs { promote: true } }
I20260926 06:20:36.810218 12367 leader_election.cc:288] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7490610226774824aaae08390ef4c1e1 (127.4.202.190:34455)
I20260926 06:20:36.810660 12368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:36.810419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.0.0.1:50476)
0926 06:20:36.810532 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:36.810573 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":68,"threads_started":1}
I20260926 06:20:36.811197 12368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:36.811071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.0.0.1:50492)
0926 06:20:36.811108 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:36.811138 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260926 06:20:36.811311 11600 leader_election.cc:334] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7490610226774824aaae08390ef4c1e1 (127.4.202.190:34455): Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
I20260926 06:20:36.811372 11600 leader_election.cc:302] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 1245bf7b57c549d28ef3f35463029b01; no voters: 7490610226774824aaae08390ef4c1e1
I20260926 06:20:36.811462 12367 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 06:20:37.093783 12371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:37.093507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.131:47823)
0926 06:20:37.093684 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:37.093725 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":69,"threads_started":1}
I20260926 06:20:37.093861 12370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:37.093496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.129:55243)
0926 06:20:37.093776 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:37.093814 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":59,"threads_started":1}
I20260926 06:20:37.094060 12372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:37.093774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.130:44327)
0926 06:20:37.093972 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:37.094014 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":43,"threads_started":1}
I20260926 06:20:38.094691 12374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:38.094405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.131:44649)
0926 06:20:38.094596 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:38.094642 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":44,"threads_started":1}
I20260926 06:20:38.094686 12373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:38.094301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.129:48941)
0926 06:20:38.094542 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:38.094626 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":62,"threads_started":1}
I20260926 06:20:38.094898 12375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:38.094511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.130:45881)
0926 06:20:38.094816 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:38.094852 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":54,"threads_started":1}
I20260926 06:20:38.741712 12376 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7490610226774824aaae08390ef4c1e1)
I20260926 06:20:38.741804 12376 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 40149 } attrs { promote: false } } peers { permanent_uuid: "092902ed31314ba181f705bd4701e4c7" member_type: NON_VOTER last_known_addr { host: "127.4.202.188" port: 33663 } attrs { promote: true } }
I20260926 06:20:38.741992 12376 leader_election.cc:288] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7490610226774824aaae08390ef4c1e1 (127.4.202.190:34455)
I20260926 06:20:38.742319 12377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:38.742128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.0.0.1:50590)
0926 06:20:38.742230 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:38.742270 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":60,"threads_started":1}
W20260926 06:20:38.742403 11600 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20260926 06:20:38.742794 12377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:38.742696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.0.0.1:50606)
0926 06:20:38.742742 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:38.742768 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260926 06:20:38.742875 11600 leader_election.cc:334] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7490610226774824aaae08390ef4c1e1 (127.4.202.190:34455): Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
I20260926 06:20:38.742918 11600 leader_election.cc:302] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 1245bf7b57c549d28ef3f35463029b01; no voters: 7490610226774824aaae08390ef4c1e1
I20260926 06:20:38.743000 12376 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 06:20:39.095526 12379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:39.095200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.129:53841)
0926 06:20:39.095428 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:39.095472 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":82,"threads_started":1}
I20260926 06:20:39.095561 12380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:39.095346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.131:39277)
0926 06:20:39.095477 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:39.095514 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":52,"threads_started":1}
I20260926 06:20:39.095918 12381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:39.095386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.130:34249)
0926 06:20:39.095841 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:39.095876 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":42,"threads_started":1}
I20260926 06:20:40.096401 12382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:40.096074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.131:59785)
0926 06:20:40.096300 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:40.096340 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":80,"threads_started":1}
I20260926 06:20:40.096621 12383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:40.096331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.130:53289)
0926 06:20:40.096531 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:40.096572 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":50,"threads_started":1}
I20260926 06:20:40.096846 12384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:40.096527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.129:57701)
0926 06:20:40.096744 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:40.096790 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":40,"threads_started":1}
I20260926 06:20:41.097229 12385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:41.097015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.131:60273)
0926 06:20:41.097140 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:41.097178 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":60,"threads_started":1}
I20260926 06:20:41.097374 12386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:41.097127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.130:45403)
0926 06:20:41.097232 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:41.097281 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":49,"threads_started":1}
I20260926 06:20:41.097854 12387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:41.097635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.129:41377)
0926 06:20:41.097782 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:41.097813 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":65,"threads_started":1}
I20260926 06:20:42.098359 12389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:42.097930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.130:49063)
0926 06:20:42.098167 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:42.098210 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":75,"threads_started":1}
I20260926 06:20:42.098486 12388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:42.097843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.131:33093)
0926 06:20:42.098395 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:42.098437 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":64,"threads_started":1}
I20260926 06:20:42.098733 12390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:42.098523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.129:53897)
0926 06:20:42.098653 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:42.098691 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":44,"threads_started":1}
I20260926 06:20:43.099153 12391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:43.098869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.130:55183)
0926 06:20:43.098997 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:43.099036 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":61,"threads_started":1}
I20260926 06:20:43.099291 12392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:43.099048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.131:55803)
0926 06:20:43.099209 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:43.099246 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":49,"threads_started":1}
I20260926 06:20:43.099656 12393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:43.099500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.129:49193)
0926 06:20:43.099600 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:43.099624 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":47,"threads_started":1}
I20260926 06:20:44.099932 12394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:44.099656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.130:55007)
0926 06:20:44.099835 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:44.099877 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":57,"threads_started":1}
I20260926 06:20:44.100214 12395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:44.099762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.131:56973)
0926 06:20:44.100129 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:44.100165 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":59,"threads_started":1}
I20260926 06:20:44.100314 12396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:44.100095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.129:58899)
0926 06:20:44.100203 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:44.100229 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":58,"threads_started":1}
I20260926 06:20:45.100812 12397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:45.100541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.130:34591)
0926 06:20:45.100713 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:45.100754 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":58,"threads_started":1}
I20260926 06:20:45.100957 12398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:45.100627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.131:34125)
0926 06:20:45.100884 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:45.100921 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":53,"threads_started":1}
I20260926 06:20:45.100965 12399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:45.100710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.129:52079)
0926 06:20:45.100876 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:45.100914 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":77,"threads_started":1}
I20260926 06:20:46.101804 12401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:46.101473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.130:35875)
0926 06:20:46.101724 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:46.101755 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":56,"threads_started":1}
I20260926 06:20:46.101804 12400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:46.101437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.131:42945)
0926 06:20:46.101677 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:46.101726 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":62,"threads_started":1}
I20260926 06:20:46.101966 12402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:46.101648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.129:59543)
0926 06:20:46.101887 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:46.101924 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":56,"threads_started":1}
I20260926 06:20:46.606374 12403 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7490610226774824aaae08390ef4c1e1)
I20260926 06:20:46.606462 12403 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7490610226774824aaae08390ef4c1e1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34455 } } peers { permanent_uuid: "1245bf7b57c549d28ef3f35463029b01" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 40149 } attrs { promote: false } } peers { permanent_uuid: "092902ed31314ba181f705bd4701e4c7" member_type: NON_VOTER last_known_addr { host: "127.4.202.188" port: 33663 } attrs { promote: true } }
I20260926 06:20:46.606699 12403 leader_election.cc:288] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7490610226774824aaae08390ef4c1e1 (127.4.202.190:34455)
I20260926 06:20:46.607036 12404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:46.606856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.0.0.1:56632)
0926 06:20:46.606956 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:46.606999 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":59,"threads_started":1}
W20260926 06:20:46.607120 11600 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
I20260926 06:20:46.607501 12404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:46.607404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.0.0.1:56642)
0926 06:20:46.607453 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:46.607480 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260926 06:20:46.607576 11600 leader_election.cc:334] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7490610226774824aaae08390ef4c1e1 (127.4.202.190:34455): Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
I20260926 06:20:46.607649 11600 leader_election.cc:302] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 1245bf7b57c549d28ef3f35463029b01; no voters: 7490610226774824aaae08390ef4c1e1
I20260926 06:20:46.607769 12403 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1245bf7b57c549d28ef3f35463029b01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 06:20:47.102638 12407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:47.102313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.131:56817)
0926 06:20:47.102524 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:47.102564 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":42,"threads_started":1}
I20260926 06:20:47.102638 12406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:47.102250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.130:58535)
0926 06:20:47.102548 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:47.102600 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":103,"threads_started":1}
I20260926 06:20:47.102639 12408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:47.102423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.129:46641)
0926 06:20:47.102527 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:47.102565 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":60,"threads_started":1}
I20260926 06:20:48.103307 12409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:48.103071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.130:50041)
0926 06:20:48.103224 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:48.103259 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":62,"threads_started":1}
I20260926 06:20:48.103309 12411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:48.103146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.131:58937)
0926 06:20:48.103250 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:48.103275 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76,"thread_start_us":41,"threads_started":1}
I20260926 06:20:48.103318 12410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:48.103089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.129:53657)
0926 06:20:48.103190 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:48.103229 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":52,"threads_started":1}
I20260926 06:20:49.104113 12413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:49.103697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.130:33227)
0926 06:20:49.104010 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:49.104056 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":61,"threads_started":1}
I20260926 06:20:49.104312 12412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:49.103698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.131:55861)
0926 06:20:49.104231 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:49.104271 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":63,"threads_started":1}
I20260926 06:20:49.104416 12414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:49.104028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.129:56259)
0926 06:20:49.104323 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:49.104367 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":51,"threads_started":1}
I20260926 06:20:50.105059 12415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:50.104667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.131:54961)
0926 06:20:50.104954 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:50.104998 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":55,"threads_started":1}
I20260926 06:20:50.105093 12416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:50.104859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.130:54005)
0926 06:20:50.105012 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:50.105047 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":61,"threads_started":1}
I20260926 06:20:50.105108 12417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:50.104892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.129:35161)
0926 06:20:50.105044 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:50.105073 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":70,"threads_started":1}
I20260926 06:20:51.105808 12418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:51.105550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.130:35109)
0926 06:20:51.105725 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:51.105757 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":64,"threads_started":1}
I20260926 06:20:51.105808 12419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:51.105551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.131:47065)
0926 06:20:51.105717 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:51.105757 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":59,"threads_started":1}
I20260926 06:20:51.106050 12420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:51.105844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.129:50787)
0926 06:20:51.105971 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:51.106001 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":66,"threads_started":1}
I20260926 06:20:52.106542 12421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:52.106201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.130:37577)
0926 06:20:52.106437 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:52.106483 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":59,"threads_started":1}
I20260926 06:20:52.106768 12423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:52.106502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.129:55277)
0926 06:20:52.106628 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:52.106665 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":54,"threads_started":1}
I20260926 06:20:52.106920 12422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:52.106502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34455 (local address 127.4.202.131:39387)
0926 06:20:52.106742 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:52.106774 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":50,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:903: Failure
Failed
Bad status: Timed out: Could not connect to the cluster: ConnectToClusterRpc(addrs: 127.4.202.190:34455,127.4.202.189:40149,127.4.202.188:33663, num_attempts: 243) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.4.202.190:34455,127.4.202.189:40149,127.4.202.188:33663, num_attempts: 1)
I20260926 06:20:52.260699 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 11762
I20260926 06:20:52.265395 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 11987
I20260926 06:20:52.269609 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 12123
I20260926 06:20:52.274657 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 11479
I20260926 06:20:52.279207 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 11668
2026-09-26T06:20:52Z chronyd exiting
I20260926 06:20:52.294831 4906 test_util.cc:175] -----------------------------------------------
I20260926 06:20:52.294889 4906 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790403591640726-4906-0
[ FAILED ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (32997 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:20:52 dist-test-slave-pd0t krb5kdc[12426](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:20:52 dist-test-slave-pd0t krb5kdc[12426](info): set up 2 sockets
Sep 26 06:20:52 dist-test-slave-pd0t krb5kdc[12426](info): commencing operation
krb5kdc: starting...
W20260926 06:20:53.299106 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:20:53 dist-test-slave-pd0t krb5kdc[12426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:20:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:20:53Z Disabled control of system clock
WARNING: no policy specified for oryx/127.4.202.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.202.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.202.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.190.keytab.
Entry for principal oryx/127.4.202.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.190.keytab.
I20260926 06:20:53.409757 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:42179
--webserver_interface=127.4.202.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.190.keytab
--principal=oryx/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:44159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:42179
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:53.489662 12439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:53.489886 12439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:53.489923 12439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:53.491135 12439 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:20:53.491194 12439 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 06:20:53.491215 12439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:53.491238 12439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:53.491268 12439 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:20:53.491291 12439 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:20:53.492787 12439 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:44159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:42179
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.190.keytab
--principal=oryx/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:42179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:53.493057 12439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:53.493278 12439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:53.498119 12447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:53.498131 12444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:53.498119 12445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:53.498450 12439 server_base.cc:1034] running on GCE node
I20260926 06:20:53.498708 12439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:53.498997 12439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:53.500176 12439 hybrid_clock.cc:648] HybridClock initialized: now 1790403653500174 us; error 37 us; skew 500 ppm
Sep 26 06:20:53 dist-test-slave-pd0t krb5kdc[12426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403653, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:53.502480 12439 init.cc:377] Logged in from keytab as oryx/127.4.202.190@KRBTEST.COM (short username oryx)
I20260926 06:20:53.502960 12439 webserver.cc:460] Webserver started at http://127.4.202.190:41481/ using document root <none> and password file <none>
I20260926 06:20:53.503160 12439 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:53.503206 12439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:53.503291 12439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:53.504264 12439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "b4c397f387024f17b7d62d585e95e5cf"
format_stamp: "Formatted at 2026-09-26 06:20:53 on dist-test-slave-pd0t"
server_key: "6f962c77f93941df75b194bdab530950"
server_key_iv: "17e20c7625b55e74b53fe45a1897f34f"
server_key_version: "encryptionkey@0"
I20260926 06:20:53.504554 12439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "b4c397f387024f17b7d62d585e95e5cf"
format_stamp: "Formatted at 2026-09-26 06:20:53 on dist-test-slave-pd0t"
server_key: "6f962c77f93941df75b194bdab530950"
server_key_iv: "17e20c7625b55e74b53fe45a1897f34f"
server_key_version: "encryptionkey@0"
I20260926 06:20:53.505713 12439 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:20:53.506484 12454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:53.506830 12439 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 06:20:53.506917 12439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "b4c397f387024f17b7d62d585e95e5cf"
format_stamp: "Formatted at 2026-09-26 06:20:53 on dist-test-slave-pd0t"
server_key: "6f962c77f93941df75b194bdab530950"
server_key_iv: "17e20c7625b55e74b53fe45a1897f34f"
server_key_version: "encryptionkey@0"
I20260926 06:20:53.506970 12439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:53.525394 12439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:53.527587 12439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:53.527696 12439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:53.532059 12439 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:42179
I20260926 06:20:53.532115 12516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:42179 every 8 connection(s)
I20260926 06:20:53.532456 12439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:20:53.533121 12517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:53.533855 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 12439
I20260926 06:20:53.533967 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:20:53.534135 4906 external_mini_cluster.cc:1428] Setting key 45bc065dd3136bf55f9bbe978179237a
I20260926 06:20:53.536108 12517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4c397f387024f17b7d62d585e95e5cf: Bootstrap starting.
I20260926 06:20:53.536830 12517 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4c397f387024f17b7d62d585e95e5cf: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:53.537125 12517 log.cc:824] T 00000000000000000000000000000000 P b4c397f387024f17b7d62d585e95e5cf: Log is configured to *not* fsync() on all Append() calls
I20260926 06:20:53.537825 12517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4c397f387024f17b7d62d585e95e5cf: No bootstrap required, opened a new log
I20260926 06:20:53.540050 12517 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4c397f387024f17b7d62d585e95e5cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4c397f387024f17b7d62d585e95e5cf" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42179 } }
I20260926 06:20:53.540187 12517 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4c397f387024f17b7d62d585e95e5cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:53.540210 12517 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4c397f387024f17b7d62d585e95e5cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4c397f387024f17b7d62d585e95e5cf, State: Initialized, Role: FOLLOWER
I20260926 06:20:53.540339 12517 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4c397f387024f17b7d62d585e95e5cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4c397f387024f17b7d62d585e95e5cf" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42179 } }
I20260926 06:20:53.540416 12517 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4c397f387024f17b7d62d585e95e5cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:20:53.540437 12517 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4c397f387024f17b7d62d585e95e5cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:20:53.540503 12517 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4c397f387024f17b7d62d585e95e5cf [term 0 FOLLOWER]: Advancing to term 1
Sep 26 06:20:53 dist-test-slave-pd0t krb5kdc[12426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM
I20260926 06:20:53.540972 12517 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4c397f387024f17b7d62d585e95e5cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4c397f387024f17b7d62d585e95e5cf" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42179 } }
I20260926 06:20:53.541113 12517 leader_election.cc:302] T 00000000000000000000000000000000 P b4c397f387024f17b7d62d585e95e5cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4c397f387024f17b7d62d585e95e5cf; no voters:
I20260926 06:20:53.541271 12517 leader_election.cc:288] T 00000000000000000000000000000000 P b4c397f387024f17b7d62d585e95e5cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:20:53.541309 12522 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4c397f387024f17b7d62d585e95e5cf [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:20:53.541435 12522 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4c397f387024f17b7d62d585e95e5cf [term 1 LEADER]: Becoming Leader. State: Replica: b4c397f387024f17b7d62d585e95e5cf, State: Running, Role: LEADER
I20260926 06:20:53.541543 12517 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4c397f387024f17b7d62d585e95e5cf [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:20:53.541602 12522 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4c397f387024f17b7d62d585e95e5cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4c397f387024f17b7d62d585e95e5cf" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42179 } }
I20260926 06:20:53.542243 12522 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4c397f387024f17b7d62d585e95e5cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4c397f387024f17b7d62d585e95e5cf. Latest consensus state: current_term: 1 leader_uuid: "b4c397f387024f17b7d62d585e95e5cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4c397f387024f17b7d62d585e95e5cf" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42179 } } }
I20260926 06:20:53.542249 12524 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4c397f387024f17b7d62d585e95e5cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4c397f387024f17b7d62d585e95e5cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4c397f387024f17b7d62d585e95e5cf" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42179 } } }
I20260926 06:20:53.542658 12524 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4c397f387024f17b7d62d585e95e5cf [sys.catalog]: This master's current role is: LEADER
I20260926 06:20:53.542658 12522 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4c397f387024f17b7d62d585e95e5cf [sys.catalog]: This master's current role is: LEADER
I20260926 06:20:53.542970 12531 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:20:53.543462 12531 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:20:53.544902 12531 catalog_manager.cc:1347] Generated new cluster ID: 59967d8f0dc944d0a0065d856944b1a3
I20260926 06:20:53.544956 12531 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:20:53.545632 12520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:53.534652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50258 (local address 127.4.202.190:42179)
0926 06:20:53.534830 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:53.534840 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:53.534871 (+ 31us) server_negotiation.cc:407] Connection header received
0926 06:20:53.535378 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:53.535386 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:53.535456 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:53.535610 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:53.536193 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:53.537593 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:53.538322 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:53.539040 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:53.539162 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:53.541548 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:53.541552 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:53.541552 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:53.541563 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:53.544053 (+ 2490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:53.544443 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:53.544446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:53.544447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:53.544546 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:53.544773 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:53.544774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:53.544775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:53.545064 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:53.545241 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:53.545316 (+ 75us) server_negotiation.cc:299] Negotiation successful
0926 06:20:53.545445 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":78,"threads_started":1}
I20260926 06:20:53.563423 12531 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:20:53.563966 12531 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:20:53.573297 12531 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4c397f387024f17b7d62d585e95e5cf: Generated new TSK 0
I20260926 06:20:53.573483 12531 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.4.202.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.202.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.202.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.129.keytab.
Entry for principal oryx/127.4.202.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.129.keytab.
I20260926 06:20:53.601411 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.129:0
--local_ip_for_outbound_sockets=127.4.202.129
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:42179
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.129.keytab
--principal=oryx/127.4.202.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:44159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:53.680655 12543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:53.680862 12543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:53.680898 12543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:53.682037 12543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:53.682097 12543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:53.682129 12543 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 06:20:53.682166 12543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.129
I20260926 06:20:53.683694 12543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:44159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.129.keytab
--principal=oryx/127.4.202.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.202.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.202.190:42179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.129
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:53.683940 12543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:53.684160 12543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:53.688802 12551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:53.688895 12543 server_base.cc:1034] running on GCE node
W20260926 06:20:53.688839 12549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:53.688840 12548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:53.689325 12543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:53.689571 12543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:53.690724 12543 hybrid_clock.cc:648] HybridClock initialized: now 1790403653690717 us; error 29 us; skew 500 ppm
Sep 26 06:20:53 dist-test-slave-pd0t krb5kdc[12426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403653, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:53.693048 12543 init.cc:377] Logged in from keytab as oryx/127.4.202.129@KRBTEST.COM (short username oryx)
I20260926 06:20:53.693485 12543 webserver.cc:460] Webserver started at http://127.4.202.129:44317/ using document root <none> and password file <none>
I20260926 06:20:53.693691 12543 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:53.693755 12543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:53.693888 12543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:53.694897 12543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-0/data/instance:
uuid: "ef281d65c9354fc483333fba732d64b4"
format_stamp: "Formatted at 2026-09-26 06:20:53 on dist-test-slave-pd0t"
server_key: "86787f7f247814d1654542e6f498913e"
server_key_iv: "4cd28dd66d0177baa5b046b7dddbb178"
server_key_version: "encryptionkey@0"
I20260926 06:20:53.695238 12543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance:
uuid: "ef281d65c9354fc483333fba732d64b4"
format_stamp: "Formatted at 2026-09-26 06:20:53 on dist-test-slave-pd0t"
server_key: "86787f7f247814d1654542e6f498913e"
server_key_iv: "4cd28dd66d0177baa5b046b7dddbb178"
server_key_version: "encryptionkey@0"
I20260926 06:20:53.696592 12543 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:20:53.697273 12558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:53.697439 12543 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:20:53.697535 12543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-0/wal
uuid: "ef281d65c9354fc483333fba732d64b4"
format_stamp: "Formatted at 2026-09-26 06:20:53 on dist-test-slave-pd0t"
server_key: "86787f7f247814d1654542e6f498913e"
server_key_iv: "4cd28dd66d0177baa5b046b7dddbb178"
server_key_version: "encryptionkey@0"
I20260926 06:20:53.697613 12543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:53.705650 12543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:53.707441 12543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:53.707595 12543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:53.708050 12543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:20:53.708114 12543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:53.708153 12543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:20:53.708201 12543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 06:20:53 dist-test-slave-pd0t krb5kdc[12426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403653, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM
I20260926 06:20:53.715353 12543 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.129:45759
I20260926 06:20:53.715442 12673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.129:45759 every 8 connection(s)
I20260926 06:20:53.715761 12543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
I20260926 06:20:53.718079 12520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:53.708570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:54995 (local address 127.4.202.190:42179)
0926 06:20:53.708640 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:53.708642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:53.708852 (+ 210us) server_negotiation.cc:407] Connection header received
0926 06:20:53.709312 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:53.709314 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:53.709327 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:53.709423 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:53.710237 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:53.711121 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:53.712177 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:53.712508 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:53.712577 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:53.715292 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:53.715293 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:53.715294 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:53.715302 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:53.716885 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:53.717278 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:53.717279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:53.717280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:53.717334 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:53.717580 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:53.717581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:53.717581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:53.717681 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:53.717731 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:53.717951 (+ 220us) server_negotiation.cc:299] Negotiation successful
0926 06:20:53.718017 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 06:20:53.718242 12567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:53.708557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42179 (local address 127.4.202.129:54995)
0926 06:20:53.708711 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:53.708748 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:53.709216 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:53.709493 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:53.709497 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:53.709847 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:53.710119 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:53.710126 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:53.711254 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:53.711255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:53.712067 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:53.712069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:53.712621 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:53.712623 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:53.712802 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:53.713361 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:53.713372 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:53.715178 (+ 1806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:53.716974 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:53.716990 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:53.716992 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:53.717203 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:53.717428 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:53.717428 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:53.717429 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:53.717522 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:53.717736 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:53.717740 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:53.717850 (+ 110us) client_negotiation.cc:769] Sending connection context
0926 06:20:53.717928 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 06:20:53.718033 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260926 06:20:53.718735 12674 heartbeater.cc:347] Connected to a master server at 127.4.202.190:42179
I20260926 06:20:53.718811 12674 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:53.718973 12674 heartbeater.cc:511] Master 127.4.202.190:42179 requested a full tablet report, sending...
I20260926 06:20:53.719508 12470 ts_manager.cc:194] Registered new tserver with Master: ef281d65c9354fc483333fba732d64b4 (127.4.202.129:45759)
I20260926 06:20:53.720427 12470 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.202.129@KRBTEST.COM'} at 127.4.202.129:54995
I20260926 06:20:53.724879 12680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:53.719647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42179 (local address 127.4.202.129:35735)
0926 06:20:53.719784 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:53.719795 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:53.719861 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:53.719975 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:53.719976 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:53.720148 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:53.720219 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:53.720221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:53.720901 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:53.720902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:53.721424 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:53.721426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:53.721786 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:53.721787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:53.721847 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:53.722170 (+ 323us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:53.722173 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:53.722471 (+ 298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:53.723763 (+ 1292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:53.723764 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:53.723765 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:53.723944 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:53.724163 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:53.724164 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:53.724164 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:53.724219 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:53.724485 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:53.724486 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:53.724535 (+ 49us) client_negotiation.cc:769] Sending connection context
0926 06:20:53.724630 (+ 95us) client_negotiation.cc:240] Negotiation successful
0926 06:20:53.724752 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":26,"thread_start_us":73,"threads_started":1}
I20260926 06:20:53.724872 12520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:53.719690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:35735 (local address 127.4.202.190:42179)
0926 06:20:53.719725 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:53.719727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:53.719854 (+ 127us) server_negotiation.cc:407] Connection header received
0926 06:20:53.719916 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:53.719916 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:53.719923 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:53.719962 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:53.720280 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:53.720800 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:53.721511 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:53.721698 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:53.721783 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:53.722557 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:53.722558 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:53.722558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:53.722565 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:53.723628 (+ 1063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:53.724028 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:53.724029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:53.724029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:53.724080 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:53.724305 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:53.724306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:53.724306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:53.724418 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:53.724482 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:53.724668 (+ 186us) server_negotiation.cc:299] Negotiation successful
0926 06:20:53.724751 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 06:20:53.725862 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 12543
I20260926 06:20:53.725977 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance
I20260926 06:20:53.726163 4906 external_mini_cluster.cc:1428] Setting key ac5255550e523efb4f6f68ccdeb2bb14
WARNING: no policy specified for oryx/127.4.202.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.202.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.202.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.130.keytab.
Entry for principal oryx/127.4.202.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.130.keytab.
I20260926 06:20:53.749925 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.130:0
--local_ip_for_outbound_sockets=127.4.202.130
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:42179
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.130.keytab
--principal=oryx/127.4.202.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:44159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:53.828255 12683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:53.828433 12683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:53.828456 12683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:53.829474 12683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:53.829509 12683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:53.829528 12683 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 06:20:53.829545 12683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.130
I20260926 06:20:53.830994 12683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:44159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.130.keytab
--principal=oryx/127.4.202.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.202.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.202.190:42179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.130
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:53.831158 12683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:53.831332 12683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:53.836410 12689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:53.836431 12691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:53.836493 12683 server_base.cc:1034] running on GCE node
W20260926 06:20:53.836450 12688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:53.836774 12683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:53.837023 12683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:53.838189 12683 hybrid_clock.cc:648] HybridClock initialized: now 1790403653838177 us; error 50 us; skew 500 ppm
Sep 26 06:20:53 dist-test-slave-pd0t krb5kdc[12426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403653, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:53.840847 12683 init.cc:377] Logged in from keytab as oryx/127.4.202.130@KRBTEST.COM (short username oryx)
I20260926 06:20:53.841288 12683 webserver.cc:460] Webserver started at http://127.4.202.130:41609/ using document root <none> and password file <none>
I20260926 06:20:53.841468 12683 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:53.841529 12683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:53.841619 12683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:53.842820 12683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-1/data/instance:
uuid: "6ba18eeef9a74b97ada822d2a00cfdf3"
format_stamp: "Formatted at 2026-09-26 06:20:53 on dist-test-slave-pd0t"
server_key: "1cdab8db78404e8103d3c5839fa980b6"
server_key_iv: "842a11499df5a221301cfd7ab4e1e943"
server_key_version: "encryptionkey@0"
I20260926 06:20:53.843142 12683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance:
uuid: "6ba18eeef9a74b97ada822d2a00cfdf3"
format_stamp: "Formatted at 2026-09-26 06:20:53 on dist-test-slave-pd0t"
server_key: "1cdab8db78404e8103d3c5839fa980b6"
server_key_iv: "842a11499df5a221301cfd7ab4e1e943"
server_key_version: "encryptionkey@0"
I20260926 06:20:53.844631 12683 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260926 06:20:53.845268 12698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:53.845436 12683 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:53.845504 12683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-1/wal
uuid: "6ba18eeef9a74b97ada822d2a00cfdf3"
format_stamp: "Formatted at 2026-09-26 06:20:53 on dist-test-slave-pd0t"
server_key: "1cdab8db78404e8103d3c5839fa980b6"
server_key_iv: "842a11499df5a221301cfd7ab4e1e943"
server_key_version: "encryptionkey@0"
I20260926 06:20:53.845553 12683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:53.879318 12683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:53.881520 12683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:53.881625 12683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:53.882054 12683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:20:53.882094 12683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:53.882117 12683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:20:53.882172 12683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 06:20:53 dist-test-slave-pd0t krb5kdc[12426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403653, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM
I20260926 06:20:53.888761 12683 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.130:44939
I20260926 06:20:53.888813 12813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.130:44939 every 8 connection(s)
I20260926 06:20:53.889107 12683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
I20260926 06:20:53.891660 12520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:53.882544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:42485 (local address 127.4.202.190:42179)
0926 06:20:53.882618 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:53.882621 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:53.882819 (+ 198us) server_negotiation.cc:407] Connection header received
0926 06:20:53.883257 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:53.883259 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:53.883273 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:53.883354 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:53.884084 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:53.884855 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:53.886430 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:53.886815 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:53.886902 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:53.889277 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:53.889279 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:53.889279 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:53.889287 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:53.890563 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:53.890984 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:53.890984 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:53.890985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:53.891033 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:53.891210 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:53.891211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:53.891211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:53.891304 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:53.891352 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:53.891551 (+ 199us) server_negotiation.cc:299] Negotiation successful
0926 06:20:53.891604 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 06:20:53.891921 12707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:53.882524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42179 (local address 127.4.202.130:42485)
0926 06:20:53.882693 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:53.882730 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:53.883166 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:53.883435 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:53.883439 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:53.883720 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:53.884006 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:53.884012 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:53.884947 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:53.884948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:53.886330 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:53.886334 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:53.886900 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:53.886901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:53.886998 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:53.887480 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:53.887490 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:53.889154 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:53.890685 (+ 1531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:53.890710 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:53.890711 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:53.890919 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:53.891073 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:53.891074 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:53.891074 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:53.891164 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:53.891360 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:53.891364 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:53.891459 (+ 95us) client_negotiation.cc:769] Sending connection context
0926 06:20:53.891538 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 06:20:53.891641 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":95,"threads_started":1}
I20260926 06:20:53.892448 12814 heartbeater.cc:347] Connected to a master server at 127.4.202.190:42179
I20260926 06:20:53.892529 12814 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:53.892695 12814 heartbeater.cc:511] Master 127.4.202.190:42179 requested a full tablet report, sending...
I20260926 06:20:53.893062 12470 ts_manager.cc:194] Registered new tserver with Master: 6ba18eeef9a74b97ada822d2a00cfdf3 (127.4.202.130:44939)
I20260926 06:20:53.893469 12470 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.202.130@KRBTEST.COM'} at 127.4.202.130:42485
I20260926 06:20:53.894275 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 12683
I20260926 06:20:53.894394 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance
I20260926 06:20:53.894564 4906 external_mini_cluster.cc:1428] Setting key 36f092f1526a64ab29f9efa9b583aa9c
I20260926 06:20:53.899287 12520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:53.893444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:60651 (local address 127.4.202.190:42179)
0926 06:20:53.893650 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:53.893651 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:53.893657 (+ 6us) server_negotiation.cc:407] Connection header received
0926 06:20:53.893671 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:53.893671 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:53.893677 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:53.893738 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:53.894024 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:53.894720 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:53.895696 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:53.895869 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:53.895919 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:53.896933 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:53.896934 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:53.896934 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:53.896941 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:53.898122 (+ 1181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:53.898460 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:53.898461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:53.898461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:53.898513 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:53.898744 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:53.898745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:53.898745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:53.898899 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:53.898972 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:53.899130 (+ 158us) server_negotiation.cc:299] Negotiation successful
0926 06:20:53.899201 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193}
I20260926 06:20:53.899305 12820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:53.893376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42179 (local address 127.4.202.130:60651)
0926 06:20:53.893490 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:53.893499 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:53.893545 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:53.893743 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:53.893744 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:53.893901 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:53.893950 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:53.893952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:53.894863 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:53.894864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:53.895616 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:53.895619 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:53.895939 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:53.895940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:53.896013 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:53.896444 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:53.896449 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:53.896848 (+ 399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:53.898221 (+ 1373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:53.898222 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:53.898222 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:53.898387 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:53.898617 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:53.898618 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:53.898618 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:53.898673 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:53.898976 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:53.898977 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:53.899020 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 06:20:53.899105 (+ 85us) client_negotiation.cc:240] Negotiation successful
0926 06:20:53.899200 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":74,"threads_started":1}
WARNING: no policy specified for oryx/127.4.202.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.202.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.202.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.131.keytab.
Entry for principal oryx/127.4.202.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.131.keytab.
I20260926 06:20:53.918872 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.131:0
--local_ip_for_outbound_sockets=127.4.202.131
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:42179
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.131.keytab
--principal=oryx/127.4.202.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:44159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:53.996886 12823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:53.997112 12823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:53.997150 12823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:53.998313 12823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:53.998376 12823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:53.998410 12823 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 06:20:53.998450 12823 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.131
I20260926 06:20:53.999967 12823 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:44159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.131.keytab
--principal=oryx/127.4.202.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.202.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.202.190:42179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.131
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:54.000221 12823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:54.000470 12823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:54.005342 12829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:54.005385 12828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:54.005357 12831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:54.005498 12823 server_base.cc:1034] running on GCE node
I20260926 06:20:54.005690 12823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:54.005934 12823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:54.007110 12823 hybrid_clock.cc:648] HybridClock initialized: now 1790403654007095 us; error 25 us; skew 500 ppm
Sep 26 06:20:54 dist-test-slave-pd0t krb5kdc[12426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403654, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:54.009431 12823 init.cc:377] Logged in from keytab as oryx/127.4.202.131@KRBTEST.COM (short username oryx)
I20260926 06:20:54.009861 12823 webserver.cc:460] Webserver started at http://127.4.202.131:35347/ using document root <none> and password file <none>
I20260926 06:20:54.010048 12823 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:54.010087 12823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:54.010249 12823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:54.011250 12823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-2/data/instance:
uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c"
format_stamp: "Formatted at 2026-09-26 06:20:54 on dist-test-slave-pd0t"
server_key: "d6eee0b19dcd095fded01688a5fdec42"
server_key_iv: "65a4272760772d03b40192b0f48f8bd0"
server_key_version: "encryptionkey@0"
I20260926 06:20:54.011603 12823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance:
uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c"
format_stamp: "Formatted at 2026-09-26 06:20:54 on dist-test-slave-pd0t"
server_key: "d6eee0b19dcd095fded01688a5fdec42"
server_key_iv: "65a4272760772d03b40192b0f48f8bd0"
server_key_version: "encryptionkey@0"
I20260926 06:20:54.012902 12823 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:20:54.013621 12838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:54.013810 12823 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:20:54.013897 12823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-2/wal
uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c"
format_stamp: "Formatted at 2026-09-26 06:20:54 on dist-test-slave-pd0t"
server_key: "d6eee0b19dcd095fded01688a5fdec42"
server_key_iv: "65a4272760772d03b40192b0f48f8bd0"
server_key_version: "encryptionkey@0"
I20260926 06:20:54.013968 12823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:54.021950 12823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:54.023926 12823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:54.024071 12823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:54.024612 12823 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:20:54.024660 12823 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:54.024693 12823 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:20:54.024722 12823 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:54.031664 12823 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.131:35099
I20260926 06:20:54.031709 12953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.131:35099 every 8 connection(s)
I20260926 06:20:54.031993 12823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
I20260926 06:20:54.032888 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 12823
I20260926 06:20:54.032994 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance
I20260926 06:20:54.033169 4906 external_mini_cluster.cc:1428] Setting key fcc4ca9bb7e72375f4fa3ca28fd7c668
Sep 26 06:20:54 dist-test-slave-pd0t krb5kdc[12426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403654, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM
I20260926 06:20:54.039932 12520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.025004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:56461 (local address 127.4.202.190:42179)
0926 06:20:54.025081 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:54.025083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:54.025251 (+ 168us) server_negotiation.cc:407] Connection header received
0926 06:20:54.025673 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:54.025674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:54.025688 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:54.025766 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:54.026382 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.027000 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.028007 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.028211 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.028298 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.037449 (+ 9151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:54.037451 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:54.037451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:54.037457 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:54.038794 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:54.039152 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:54.039152 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:54.039153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:54.039199 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:54.039404 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:54.039405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:54.039405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:54.039500 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:54.039550 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:54.039818 (+ 268us) server_negotiation.cc:299] Negotiation successful
0926 06:20:54.039873 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 06:20:54.040094 12847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.024987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42179 (local address 127.4.202.131:56461)
0926 06:20:54.025126 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:54.025162 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:54.025584 (+ 422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:54.025814 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:54.025818 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:54.026028 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:54.026227 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:54.026233 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.027114 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.027115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:54.027901 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:54.027903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.028313 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.028314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:54.028434 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.036066 (+ 7632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:54.036073 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:54.037367 (+ 1294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:54.038863 (+ 1496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:54.038869 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:54.038871 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:54.039090 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:54.039253 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:54.039253 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:54.039253 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:54.039350 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:54.039581 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:54.039587 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:54.039694 (+ 107us) client_negotiation.cc:769] Sending connection context
0926 06:20:54.039794 (+ 100us) client_negotiation.cc:240] Negotiation successful
0926 06:20:54.039905 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":71,"threads_started":1}
I20260926 06:20:54.040570 12954 heartbeater.cc:347] Connected to a master server at 127.4.202.190:42179
I20260926 06:20:54.040648 12954 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:54.040834 12954 heartbeater.cc:511] Master 127.4.202.190:42179 requested a full tablet report, sending...
I20260926 06:20:54.041196 12471 ts_manager.cc:194] Registered new tserver with Master: 388fa46aa14f4d7fbfe0e887e8ec4f3c (127.4.202.131:35099)
I20260926 06:20:54.041671 12471 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.202.131@KRBTEST.COM'} at 127.4.202.131:56461
I20260926 06:20:54.044729 4906 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:20:54.047812 12960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.041563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42179 (local address 127.4.202.131:35751)
0926 06:20:54.041678 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:54.041688 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:54.041745 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:54.041891 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:54.041892 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:54.042098 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:54.042180 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:54.042182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.043041 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.043042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:54.043652 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:54.043654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.044134 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.044135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:54.044200 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.044612 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:54.044616 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:54.044975 (+ 359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:54.046769 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:54.046770 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:54.046771 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:54.046991 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:54.047177 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:54.047178 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:54.047178 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:54.047234 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:54.047487 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:54.047488 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:54.047531 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 06:20:54.047619 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 06:20:54.047715 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":64,"threads_started":1}
I20260926 06:20:54.047799 12520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.041632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:35751 (local address 127.4.202.190:42179)
0926 06:20:54.041676 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:54.041677 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:54.041751 (+ 74us) server_negotiation.cc:407] Connection header received
0926 06:20:54.041797 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:54.041798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:54.041807 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:54.041885 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:54.042278 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.042872 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.043753 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.044043 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.044124 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.045048 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:54.045049 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:54.045049 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:54.045056 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:54.046673 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:54.047048 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:54.047049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:54.047050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:54.047099 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:54.047308 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:54.047309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:54.047309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:54.047419 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:54.047496 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:54.047646 (+ 150us) server_negotiation.cc:299] Negotiation successful
0926 06:20:54.047713 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
Sep 26 06:20:54 dist-test-slave-pd0t krb5kdc[12426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403654, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 06:20:54 dist-test-slave-pd0t krb5kdc[12426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:20:54 dist-test-slave-pd0t krb5kdc[12426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
W20260926 06:20:54.065079 12520 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:20:54.060143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50272 (local address 127.4.202.190:42179)
0926 06:20:54.060195 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:54.060197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:54.060205 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:20:54.060263 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:54.060264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:54.060273 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:54.060359 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:54.060860 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.061298 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.062011 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.062178 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.062249 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.064979 (+ 2730us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50272: BlockingRecv error: recv got EOF from 127.0.0.1:50272 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 26 06:20:54 dist-test-slave-pd0t krb5kdc[12426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403654, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM
I20260926 06:20:54.072155 12520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.065811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50282 (local address 127.4.202.190:42179)
0926 06:20:54.065865 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:54.065867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:54.065920 (+ 53us) server_negotiation.cc:407] Connection header received
0926 06:20:54.065992 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:54.065993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:54.066003 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:54.066055 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:54.066567 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.067027 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.067705 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.067891 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.067975 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.069865 (+ 1890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:54.069867 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:54.069867 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:54.069874 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:54.071040 (+ 1166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:54.071381 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:54.071382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:54.071383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:54.071434 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:54.071647 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:54.071648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:54.071648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:54.071749 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:54.071859 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:54.071960 (+ 101us) server_negotiation.cc:299] Negotiation successful
0926 06:20:54.072026 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 06:20:54.073632 12471 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50282:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:20:54.075273 12471 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:20:54.081409 12980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.077595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:44939 (local address 127.0.0.1:58112)
0926 06:20:54.077715 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:54.077724 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:54.077768 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:54.078095 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:54.078096 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:54.078098 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:20:54.078154 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:54.078156 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.079527 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.079528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:54.080361 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:54.080364 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.081143 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.081144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:54.081215 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.081227 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 06:20:54.081299 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 06:20:54.081342 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":59,"threads_started":1}
I20260926 06:20:54.081589 12981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.077784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58112 (local address 127.4.202.130:44939)
0926 06:20:54.077960 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:54.077964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:54.077971 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:20:54.078000 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:54.078001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:54.078013 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:54.078098 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:20:54.078374 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.079447 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.080431 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.081062 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.081134 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.081200 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:54.081329 (+ 129us) server_negotiation.cc:299] Negotiation successful
0926 06:20:54.081365 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":94,"threads_started":1}
I20260926 06:20:54.081775 12977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.077408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:35099 (local address 127.0.0.1:57826)
0926 06:20:54.077666 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:54.077677 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:54.077769 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:54.078059 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:54.078062 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:54.078064 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:20:54.078124 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:54.078126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.079372 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.079373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:54.080590 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:54.080593 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.081530 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.081530 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:54.081592 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.081596 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:20:54.081662 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 06:20:54.081710 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":38,"thread_start_us":79,"threads_started":1}
I20260926 06:20:54.081938 12976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.077408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:45759 (local address 127.0.0.1:47408)
0926 06:20:54.077561 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:54.077595 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:54.077655 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:54.078213 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:54.078214 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:54.078216 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:20:54.078247 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:54.078249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.079254 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.079256 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:54.080790 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:54.080791 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.081691 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.081692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:54.081756 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.081760 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:20:54.081835 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 06:20:54.081875 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":30,"thread_start_us":56,"threads_started":1}
I20260926 06:20:54.082077 12978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.077503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57826 (local address 127.4.202.131:35099)
0926 06:20:54.077665 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:54.077670 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:54.077744 (+ 74us) server_negotiation.cc:407] Connection header received
0926 06:20:54.077881 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:54.077885 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:54.077902 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:54.077983 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:20:54.078186 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.079286 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.080665 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.081370 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.081436 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.081497 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:54.081948 (+ 451us) server_negotiation.cc:299] Negotiation successful
0926 06:20:54.081989 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":72,"threads_started":1}
I20260926 06:20:54.082283 12979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.077518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47408 (local address 127.4.202.129:45759)
0926 06:20:54.078007 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:54.078011 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:54.078022 (+ 11us) server_negotiation.cc:407] Connection header received
0926 06:20:54.078059 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:54.078061 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:54.078074 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:54.078365 (+ 291us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:20:54.078454 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.079168 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.080866 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.081490 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.081556 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.081650 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:54.082133 (+ 483us) server_negotiation.cc:299] Negotiation successful
0926 06:20:54.082179 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":67,"threads_started":1}
I20260926 06:20:54.082451 12748 tablet_service.cc:1467] Processing CreateTablet for tablet 7f8ef053010a45b8b62a03ba2bb4a811 (DEFAULT_TABLE table=test-table [id=58558832407d4f63a099877fa8ba43fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:20:54.082828 12748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f8ef053010a45b8b62a03ba2bb4a811. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:54.082984 12608 tablet_service.cc:1467] Processing CreateTablet for tablet 7f8ef053010a45b8b62a03ba2bb4a811 (DEFAULT_TABLE table=test-table [id=58558832407d4f63a099877fa8ba43fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:20:54.083268 12608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f8ef053010a45b8b62a03ba2bb4a811. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:54.086998 12982 tablet_bootstrap.cc:492] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3: Bootstrap starting.
I20260926 06:20:54.087196 12888 tablet_service.cc:1467] Processing CreateTablet for tablet 7f8ef053010a45b8b62a03ba2bb4a811 (DEFAULT_TABLE table=test-table [id=58558832407d4f63a099877fa8ba43fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:20:54.087525 12888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f8ef053010a45b8b62a03ba2bb4a811. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:54.087644 12982 tablet_bootstrap.cc:654] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:54.088059 12982 log.cc:824] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3: Log is configured to *not* fsync() on all Append() calls
I20260926 06:20:54.088392 12983 tablet_bootstrap.cc:492] T 7f8ef053010a45b8b62a03ba2bb4a811 P ef281d65c9354fc483333fba732d64b4: Bootstrap starting.
I20260926 06:20:54.088784 12982 tablet_bootstrap.cc:492] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3: No bootstrap required, opened a new log
I20260926 06:20:54.088867 12982 ts_tablet_manager.cc:1397] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:20:54.088986 12983 tablet_bootstrap.cc:654] T 7f8ef053010a45b8b62a03ba2bb4a811 P ef281d65c9354fc483333fba732d64b4: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:54.089306 12983 log.cc:824] T 7f8ef053010a45b8b62a03ba2bb4a811 P ef281d65c9354fc483333fba732d64b4: Log is configured to *not* fsync() on all Append() calls
I20260926 06:20:54.090129 12983 tablet_bootstrap.cc:492] T 7f8ef053010a45b8b62a03ba2bb4a811 P ef281d65c9354fc483333fba732d64b4: No bootstrap required, opened a new log
I20260926 06:20:54.090273 12983 ts_tablet_manager.cc:1397] T 7f8ef053010a45b8b62a03ba2bb4a811 P ef281d65c9354fc483333fba732d64b4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:20:54.090957 12986 tablet_bootstrap.cc:492] T 7f8ef053010a45b8b62a03ba2bb4a811 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Bootstrap starting.
I20260926 06:20:54.090986 12982 raft_consensus.cc:348] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } }
I20260926 06:20:54.091180 12982 raft_consensus.cc:374] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:54.091238 12982 raft_consensus.cc:729] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ba18eeef9a74b97ada822d2a00cfdf3, State: Initialized, Role: FOLLOWER
I20260926 06:20:54.091389 12982 consensus_queue.cc:260] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } }
I20260926 06:20:54.091571 12986 tablet_bootstrap.cc:654] T 7f8ef053010a45b8b62a03ba2bb4a811 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:54.091686 12982 ts_tablet_manager.cc:1428] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 06:20:54.091703 12814 heartbeater.cc:503] Master 127.4.202.190:42179 was elected leader, sending a full tablet report...
I20260926 06:20:54.091930 12986 log.cc:824] T 7f8ef053010a45b8b62a03ba2bb4a811 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Log is configured to *not* fsync() on all Append() calls
I20260926 06:20:54.092308 12983 raft_consensus.cc:348] T 7f8ef053010a45b8b62a03ba2bb4a811 P ef281d65c9354fc483333fba732d64b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } }
I20260926 06:20:54.092559 12983 raft_consensus.cc:374] T 7f8ef053010a45b8b62a03ba2bb4a811 P ef281d65c9354fc483333fba732d64b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:54.092613 12983 raft_consensus.cc:729] T 7f8ef053010a45b8b62a03ba2bb4a811 P ef281d65c9354fc483333fba732d64b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef281d65c9354fc483333fba732d64b4, State: Initialized, Role: FOLLOWER
I20260926 06:20:54.092650 12986 tablet_bootstrap.cc:492] T 7f8ef053010a45b8b62a03ba2bb4a811 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: No bootstrap required, opened a new log
I20260926 06:20:54.092741 12986 ts_tablet_manager.cc:1397] T 7f8ef053010a45b8b62a03ba2bb4a811 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:20:54.092756 12983 consensus_queue.cc:260] T 7f8ef053010a45b8b62a03ba2bb4a811 P ef281d65c9354fc483333fba732d64b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } }
I20260926 06:20:54.093072 12983 ts_tablet_manager.cc:1428] T 7f8ef053010a45b8b62a03ba2bb4a811 P ef281d65c9354fc483333fba732d64b4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 06:20:54.093197 12674 heartbeater.cc:503] Master 127.4.202.190:42179 was elected leader, sending a full tablet report...
I20260926 06:20:54.094415 12986 raft_consensus.cc:348] T 7f8ef053010a45b8b62a03ba2bb4a811 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } }
I20260926 06:20:54.094552 12986 raft_consensus.cc:374] T 7f8ef053010a45b8b62a03ba2bb4a811 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:54.094590 12986 raft_consensus.cc:729] T 7f8ef053010a45b8b62a03ba2bb4a811 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 388fa46aa14f4d7fbfe0e887e8ec4f3c, State: Initialized, Role: FOLLOWER
I20260926 06:20:54.094722 12986 consensus_queue.cc:260] T 7f8ef053010a45b8b62a03ba2bb4a811 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } }
I20260926 06:20:54.094983 12986 ts_tablet_manager.cc:1428] T 7f8ef053010a45b8b62a03ba2bb4a811 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 06:20:54.095069 12954 heartbeater.cc:503] Master 127.4.202.190:42179 was elected leader, sending a full tablet report...
W20260926 06:20:54.139652 12815 tablet.cc:2369] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:20:54.171891 12987 raft_consensus.cc:482] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:20:54.172034 12987 raft_consensus.cc:504] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } }
I20260926 06:20:54.172389 12987 leader_election.cc:288] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef281d65c9354fc483333fba732d64b4 (127.4.202.129:45759), 388fa46aa14f4d7fbfe0e887e8ec4f3c (127.4.202.131:35099)
I20260926 06:20:54.176133 12978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.172665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:47375 (local address 127.4.202.131:35099)
0926 06:20:54.172709 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:54.172711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:54.173011 (+ 300us) server_negotiation.cc:407] Connection header received
0926 06:20:54.173044 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:54.173045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:54.173056 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:54.173112 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:20:54.173252 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.173925 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.175240 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.175797 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.175874 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.175882 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:54.176055 (+ 173us) server_negotiation.cc:299] Negotiation successful
0926 06:20:54.176085 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 06:20:54.176216 12991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.172591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:35099 (local address 127.4.202.130:47375)
0926 06:20:54.172931 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:54.172941 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:54.172997 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:54.173123 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:54.173124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:54.173125 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:20:54.173183 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:54.173184 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.174003 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.174004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:54.175125 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:54.175127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.175885 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.175886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:54.175979 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.175983 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:20:54.176043 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 06:20:54.176093 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":145,"thread_start_us":118,"threads_started":1}
I20260926 06:20:54.176534 12707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.172591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:45759 (local address 127.4.202.130:48399)
0926 06:20:54.172636 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:54.172647 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:54.172688 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:54.172864 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:54.172865 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:54.172867 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:20:54.172945 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:54.172947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.173897 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.173898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:54.175124 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:54.175127 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.175885 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.175886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:54.176367 (+ 481us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.176370 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:20:54.176425 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 06:20:54.176464 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":2}
I20260926 06:20:54.176549 12979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.172681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:48399 (local address 127.4.202.129:45759)
0926 06:20:54.172725 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:54.172727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:54.172735 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:20:54.172761 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:54.172761 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:54.172769 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:54.172840 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:20:54.173061 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.173821 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.175247 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.175796 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.175874 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.175882 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:54.176444 (+ 562us) server_negotiation.cc:299] Negotiation successful
0926 06:20:54.176488 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 06:20:54.176596 12908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f8ef053010a45b8b62a03ba2bb4a811" candidate_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" is_pre_election: true
I20260926 06:20:54.176734 12908 raft_consensus.cc:2454] T 7f8ef053010a45b8b62a03ba2bb4a811 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ba18eeef9a74b97ada822d2a00cfdf3 in term 0.
I20260926 06:20:54.176877 12628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f8ef053010a45b8b62a03ba2bb4a811" candidate_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef281d65c9354fc483333fba732d64b4" is_pre_election: true
I20260926 06:20:54.176936 12702 leader_election.cc:302] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 388fa46aa14f4d7fbfe0e887e8ec4f3c, 6ba18eeef9a74b97ada822d2a00cfdf3; no voters:
I20260926 06:20:54.177037 12628 raft_consensus.cc:2454] T 7f8ef053010a45b8b62a03ba2bb4a811 P ef281d65c9354fc483333fba732d64b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ba18eeef9a74b97ada822d2a00cfdf3 in term 0.
I20260926 06:20:54.177050 12987 raft_consensus.cc:2781] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:20:54.177103 12987 raft_consensus.cc:482] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:20:54.177126 12987 raft_consensus.cc:3037] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:54.177824 12987 raft_consensus.cc:504] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } }
I20260926 06:20:54.177958 12987 leader_election.cc:288] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3 [CANDIDATE]: Term 1 election: Requested vote from peers ef281d65c9354fc483333fba732d64b4 (127.4.202.129:45759), 388fa46aa14f4d7fbfe0e887e8ec4f3c (127.4.202.131:35099)
I20260926 06:20:54.178102 12628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f8ef053010a45b8b62a03ba2bb4a811" candidate_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef281d65c9354fc483333fba732d64b4"
I20260926 06:20:54.178123 12908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f8ef053010a45b8b62a03ba2bb4a811" candidate_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c"
I20260926 06:20:54.178191 12908 raft_consensus.cc:3037] T 7f8ef053010a45b8b62a03ba2bb4a811 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:54.178197 12628 raft_consensus.cc:3037] T 7f8ef053010a45b8b62a03ba2bb4a811 P ef281d65c9354fc483333fba732d64b4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:54.179029 12908 raft_consensus.cc:2454] T 7f8ef053010a45b8b62a03ba2bb4a811 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ba18eeef9a74b97ada822d2a00cfdf3 in term 1.
I20260926 06:20:54.179061 12628 raft_consensus.cc:2454] T 7f8ef053010a45b8b62a03ba2bb4a811 P ef281d65c9354fc483333fba732d64b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ba18eeef9a74b97ada822d2a00cfdf3 in term 1.
I20260926 06:20:54.179198 12702 leader_election.cc:302] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 388fa46aa14f4d7fbfe0e887e8ec4f3c, 6ba18eeef9a74b97ada822d2a00cfdf3; no voters:
I20260926 06:20:54.179302 12987 raft_consensus.cc:2781] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:20:54.179468 12987 raft_consensus.cc:686] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 1 LEADER]: Becoming Leader. State: Replica: 6ba18eeef9a74b97ada822d2a00cfdf3, State: Running, Role: LEADER
I20260926 06:20:54.179558 12987 consensus_queue.cc:237] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } }
I20260926 06:20:54.180316 12471 catalog_manager.cc:5495] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ba18eeef9a74b97ada822d2a00cfdf3 (127.4.202.130). New cstate: current_term: 1 leader_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } health_report { overall_health: HEALTHY } } }
I20260926 06:20:54.199878 12981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.197665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58124 (local address 127.4.202.130:44939)
0926 06:20:54.197717 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:54.197719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:54.197751 (+ 32us) server_negotiation.cc:407] Connection header received
0926 06:20:54.197784 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:54.197784 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:54.197813 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:54.197863 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:20:54.198000 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.198412 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.199161 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.199411 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.199486 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.199617 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:20:54.199709 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:20:54.199773 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:54.199813 (+ 40us) server_negotiation.cc:299] Negotiation successful
0926 06:20:54.199830 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 06:20:54.201498 12628 raft_consensus.cc:1261] T 7f8ef053010a45b8b62a03ba2bb4a811 P ef281d65c9354fc483333fba732d64b4 [term 1 FOLLOWER]: Refusing update from remote peer 6ba18eeef9a74b97ada822d2a00cfdf3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:20:54.201503 12908 raft_consensus.cc:1261] T 7f8ef053010a45b8b62a03ba2bb4a811 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 1 FOLLOWER]: Refusing update from remote peer 6ba18eeef9a74b97ada822d2a00cfdf3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:20:54.201819 12992 consensus_queue.cc:1035] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:20:54.201951 12987 consensus_queue.cc:1035] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:20:54.204821 12997 mvcc.cc:205] Tried to move back new op lower bound from 7333493367606894592 to 7333493367520022528. Current Snapshot: MvccSnapshot[applied={T|T < 7333493367606894592 or (T in {7333493367606894592})}]
I20260926 06:20:54.206363 12470 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:20:54.206456 12470 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:20:54.207244 12470 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b4c397f387024f17b7d62d585e95e5cf: Sending DeleteTablet for 3 replicas of tablet 7f8ef053010a45b8b62a03ba2bb4a811
I20260926 06:20:54.207545 12608 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f8ef053010a45b8b62a03ba2bb4a811 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:20:54 UTC) from {username='oryx', principal='oryx/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:47408
I20260926 06:20:54.207580 12888 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f8ef053010a45b8b62a03ba2bb4a811 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:20:54 UTC) from {username='oryx', principal='oryx/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:57826
I20260926 06:20:54.207621 12748 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f8ef053010a45b8b62a03ba2bb4a811 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:20:54 UTC) from {username='oryx', principal='oryx/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:58112
I20260926 06:20:54.207810 13006 tablet_replica.cc:330] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3: stopping tablet replica
I20260926 06:20:54.207902 13006 raft_consensus.cc:2229] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:20:54.208005 13004 tablet_replica.cc:330] T 7f8ef053010a45b8b62a03ba2bb4a811 P ef281d65c9354fc483333fba732d64b4: stopping tablet replica
I20260926 06:20:54.208041 12470 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50282:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 06:20:54.208123 13004 raft_consensus.cc:2229] T 7f8ef053010a45b8b62a03ba2bb4a811 P ef281d65c9354fc483333fba732d64b4 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260926 06:20:54.208176 12470 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:20:54.208209 13004 raft_consensus.cc:2258] T 7f8ef053010a45b8b62a03ba2bb4a811 P ef281d65c9354fc483333fba732d64b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:20:54.208463 13006 raft_consensus.cc:2258] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:20:54.208631 13004 ts_tablet_manager.cc:1905] T 7f8ef053010a45b8b62a03ba2bb4a811 P ef281d65c9354fc483333fba732d64b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:20:54.209339 12888 tablet_service.cc:1467] Processing CreateTablet for tablet 5aef9fa2349e46f8acbb93581f457580 (DEFAULT_TABLE table=test-table [id=643cb8a5f16f41198a258dd8b949f1d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:20:54.209357 12608 tablet_service.cc:1467] Processing CreateTablet for tablet 5aef9fa2349e46f8acbb93581f457580 (DEFAULT_TABLE table=test-table [id=643cb8a5f16f41198a258dd8b949f1d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:20:54.209343 12748 tablet_service.cc:1467] Processing CreateTablet for tablet 5aef9fa2349e46f8acbb93581f457580 (DEFAULT_TABLE table=test-table [id=643cb8a5f16f41198a258dd8b949f1d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:20:54.209451 12888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5aef9fa2349e46f8acbb93581f457580. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:54.209456 12608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5aef9fa2349e46f8acbb93581f457580. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:54.209517 12748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5aef9fa2349e46f8acbb93581f457580. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:54.211251 12986 tablet_bootstrap.cc:492] T 5aef9fa2349e46f8acbb93581f457580 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Bootstrap starting.
I20260926 06:20:54.211632 12986 tablet_bootstrap.cc:654] T 5aef9fa2349e46f8acbb93581f457580 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:54.211825 12982 tablet_bootstrap.cc:492] T 5aef9fa2349e46f8acbb93581f457580 P 6ba18eeef9a74b97ada822d2a00cfdf3: Bootstrap starting.
I20260926 06:20:54.212212 12982 tablet_bootstrap.cc:654] T 5aef9fa2349e46f8acbb93581f457580 P 6ba18eeef9a74b97ada822d2a00cfdf3: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:54.213089 12983 tablet_bootstrap.cc:492] T 5aef9fa2349e46f8acbb93581f457580 P ef281d65c9354fc483333fba732d64b4: Bootstrap starting.
I20260926 06:20:54.213014 13004 ts_tablet_manager.cc:1918] T 7f8ef053010a45b8b62a03ba2bb4a811 P ef281d65c9354fc483333fba732d64b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:20:54.213219 13004 log.cc:1196] T 7f8ef053010a45b8b62a03ba2bb4a811 P ef281d65c9354fc483333fba732d64b4: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-0/wal/wals/7f8ef053010a45b8b62a03ba2bb4a811
I20260926 06:20:54.213275 13006 ts_tablet_manager.cc:1905] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:20:54.213479 13004 ts_tablet_manager.cc:1939] T 7f8ef053010a45b8b62a03ba2bb4a811 P ef281d65c9354fc483333fba732d64b4: Deleting consensus metadata
I20260926 06:20:54.213543 12983 tablet_bootstrap.cc:654] T 5aef9fa2349e46f8acbb93581f457580 P ef281d65c9354fc483333fba732d64b4: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:54.214000 12456 catalog_manager.cc:4841] TS ef281d65c9354fc483333fba732d64b4 (127.4.202.129:45759): tablet 7f8ef053010a45b8b62a03ba2bb4a811 (table test-table [id=58558832407d4f63a099877fa8ba43fc]) successfully deleted
I20260926 06:20:54.214367 12983 tablet_bootstrap.cc:492] T 5aef9fa2349e46f8acbb93581f457580 P ef281d65c9354fc483333fba732d64b4: No bootstrap required, opened a new log
I20260926 06:20:54.214423 12983 ts_tablet_manager.cc:1397] T 5aef9fa2349e46f8acbb93581f457580 P ef281d65c9354fc483333fba732d64b4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 06:20:54.214511 12986 tablet_bootstrap.cc:492] T 5aef9fa2349e46f8acbb93581f457580 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: No bootstrap required, opened a new log
I20260926 06:20:54.214557 12986 ts_tablet_manager.cc:1397] T 5aef9fa2349e46f8acbb93581f457580 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260926 06:20:54.214649 12983 raft_consensus.cc:348] T 5aef9fa2349e46f8acbb93581f457580 P ef281d65c9354fc483333fba732d64b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:54.214720 13006 ts_tablet_manager.cc:1918] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:20:54.214735 12983 raft_consensus.cc:374] T 5aef9fa2349e46f8acbb93581f457580 P ef281d65c9354fc483333fba732d64b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:54.214758 12983 raft_consensus.cc:729] T 5aef9fa2349e46f8acbb93581f457580 P ef281d65c9354fc483333fba732d64b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef281d65c9354fc483333fba732d64b4, State: Initialized, Role: FOLLOWER
I20260926 06:20:54.214780 13006 log.cc:1196] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-1/wal/wals/7f8ef053010a45b8b62a03ba2bb4a811
I20260926 06:20:54.214732 12986 raft_consensus.cc:348] T 5aef9fa2349e46f8acbb93581f457580 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:54.214800 12983 consensus_queue.cc:260] T 5aef9fa2349e46f8acbb93581f457580 P ef281d65c9354fc483333fba732d64b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:54.214838 12986 raft_consensus.cc:374] T 5aef9fa2349e46f8acbb93581f457580 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:54.214864 12986 raft_consensus.cc:729] T 5aef9fa2349e46f8acbb93581f457580 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 388fa46aa14f4d7fbfe0e887e8ec4f3c, State: Initialized, Role: FOLLOWER
I20260926 06:20:54.214895 12983 ts_tablet_manager.cc:1428] T 5aef9fa2349e46f8acbb93581f457580 P ef281d65c9354fc483333fba732d64b4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 06:20:54.214908 12986 consensus_queue.cc:260] T 5aef9fa2349e46f8acbb93581f457580 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:54.215003 12986 ts_tablet_manager.cc:1428] T 5aef9fa2349e46f8acbb93581f457580 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:54.215023 13006 ts_tablet_manager.cc:1939] T 7f8ef053010a45b8b62a03ba2bb4a811 P 6ba18eeef9a74b97ada822d2a00cfdf3: Deleting consensus metadata
I20260926 06:20:54.215149 13005 tablet_replica.cc:330] T 7f8ef053010a45b8b62a03ba2bb4a811 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: stopping tablet replica
I20260926 06:20:54.215260 13005 raft_consensus.cc:2229] T 7f8ef053010a45b8b62a03ba2bb4a811 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:20:54.215373 13005 raft_consensus.cc:2258] T 7f8ef053010a45b8b62a03ba2bb4a811 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:20:54.215829 13005 ts_tablet_manager.cc:1905] T 7f8ef053010a45b8b62a03ba2bb4a811 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:20:54.215858 12458 catalog_manager.cc:4841] TS 6ba18eeef9a74b97ada822d2a00cfdf3 (127.4.202.130:44939): tablet 7f8ef053010a45b8b62a03ba2bb4a811 (table test-table [id=58558832407d4f63a099877fa8ba43fc]) successfully deleted
I20260926 06:20:54.216290 12982 tablet_bootstrap.cc:492] T 5aef9fa2349e46f8acbb93581f457580 P 6ba18eeef9a74b97ada822d2a00cfdf3: No bootstrap required, opened a new log
I20260926 06:20:54.216383 12982 ts_tablet_manager.cc:1397] T 5aef9fa2349e46f8acbb93581f457580 P 6ba18eeef9a74b97ada822d2a00cfdf3: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
W20260926 06:20:54.216468 12675 tablet.cc:2369] T 5aef9fa2349e46f8acbb93581f457580 P ef281d65c9354fc483333fba732d64b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:20:54.216558 12982 raft_consensus.cc:348] T 5aef9fa2349e46f8acbb93581f457580 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:54.216632 12982 raft_consensus.cc:374] T 5aef9fa2349e46f8acbb93581f457580 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:54.216661 12982 raft_consensus.cc:729] T 5aef9fa2349e46f8acbb93581f457580 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ba18eeef9a74b97ada822d2a00cfdf3, State: Initialized, Role: FOLLOWER
I20260926 06:20:54.216702 12982 consensus_queue.cc:260] T 5aef9fa2349e46f8acbb93581f457580 P 6ba18eeef9a74b97ada822d2a00cfdf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:54.216800 12982 ts_tablet_manager.cc:1428] T 5aef9fa2349e46f8acbb93581f457580 P 6ba18eeef9a74b97ada822d2a00cfdf3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:54.217538 13005 ts_tablet_manager.cc:1918] T 7f8ef053010a45b8b62a03ba2bb4a811 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:20:54.217609 13005 log.cc:1196] T 7f8ef053010a45b8b62a03ba2bb4a811 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-2/wal/wals/7f8ef053010a45b8b62a03ba2bb4a811
I20260926 06:20:54.217873 13005 ts_tablet_manager.cc:1939] T 7f8ef053010a45b8b62a03ba2bb4a811 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Deleting consensus metadata
I20260926 06:20:54.218196 12458 catalog_manager.cc:4841] TS 388fa46aa14f4d7fbfe0e887e8ec4f3c (127.4.202.131:35099): tablet 7f8ef053010a45b8b62a03ba2bb4a811 (table test-table [id=58558832407d4f63a099877fa8ba43fc]) successfully deleted
W20260926 06:20:54.282646 12955 tablet.cc:2369] T 5aef9fa2349e46f8acbb93581f457580 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:20:54.381937 12990 raft_consensus.cc:482] T 5aef9fa2349e46f8acbb93581f457580 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:20:54.382053 12990 raft_consensus.cc:504] T 5aef9fa2349e46f8acbb93581f457580 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:54.382450 12990 leader_election.cc:288] T 5aef9fa2349e46f8acbb93581f457580 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef281d65c9354fc483333fba732d64b4 (127.4.202.129:45759), 6ba18eeef9a74b97ada822d2a00cfdf3 (127.4.202.130:44939)
I20260926 06:20:54.386390 12979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.382717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:41675 (local address 127.4.202.129:45759)
0926 06:20:54.382769 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:54.382771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:54.382897 (+ 126us) server_negotiation.cc:407] Connection header received
0926 06:20:54.382957 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:54.382958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:54.382968 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:54.383018 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:20:54.383181 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.383932 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.385476 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.386024 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.386102 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.386111 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:54.386290 (+ 179us) server_negotiation.cc:299] Negotiation successful
0926 06:20:54.386324 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260926 06:20:54.386400 12847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.382669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:44939 (local address 127.4.202.131:43187)
0926 06:20:54.382722 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:54.382735 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:54.382786 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:54.383009 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:54.383010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:54.383012 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:20:54.383097 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:54.383099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.384016 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.384017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:54.385363 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:54.385366 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.386117 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.386118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:54.386201 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.386207 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 06:20:54.386267 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 06:20:54.386303 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":99}
I20260926 06:20:54.386416 12981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.382744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:43187 (local address 127.4.202.130:44939)
0926 06:20:54.382788 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:54.382790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:54.382799 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:20:54.382893 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:54.382894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:54.382908 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:54.382963 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:20:54.383184 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.383925 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.385476 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.386024 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.386102 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.386111 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:54.386291 (+ 180us) server_negotiation.cc:299] Negotiation successful
0926 06:20:54.386355 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 06:20:54.386426 13010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.382722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:45759 (local address 127.4.202.131:41675)
0926 06:20:54.382825 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:54.382835 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:54.382898 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:54.383028 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:54.383029 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:54.383030 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:20:54.383098 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:54.383099 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.384016 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.384017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:54.385375 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:54.385377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.386117 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.386118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:54.386201 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.386208 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 06:20:54.386267 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 06:20:54.386333 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":58,"thread_start_us":55,"threads_started":1}
I20260926 06:20:54.386667 12628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5aef9fa2349e46f8acbb93581f457580" candidate_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef281d65c9354fc483333fba732d64b4" is_pre_election: true
I20260926 06:20:54.386713 12768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5aef9fa2349e46f8acbb93581f457580" candidate_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" is_pre_election: true
I20260926 06:20:54.386771 12628 raft_consensus.cc:2454] T 5aef9fa2349e46f8acbb93581f457580 P ef281d65c9354fc483333fba732d64b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 388fa46aa14f4d7fbfe0e887e8ec4f3c in term 0.
I20260926 06:20:54.386838 12768 raft_consensus.cc:2454] T 5aef9fa2349e46f8acbb93581f457580 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 388fa46aa14f4d7fbfe0e887e8ec4f3c in term 0.
I20260926 06:20:54.386976 12840 leader_election.cc:302] T 5aef9fa2349e46f8acbb93581f457580 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 388fa46aa14f4d7fbfe0e887e8ec4f3c, ef281d65c9354fc483333fba732d64b4; no voters:
I20260926 06:20:54.387130 12990 raft_consensus.cc:2781] T 5aef9fa2349e46f8acbb93581f457580 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:20:54.387176 12990 raft_consensus.cc:482] T 5aef9fa2349e46f8acbb93581f457580 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:20:54.387197 12990 raft_consensus.cc:3037] T 5aef9fa2349e46f8acbb93581f457580 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:54.387828 12990 raft_consensus.cc:504] T 5aef9fa2349e46f8acbb93581f457580 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:54.387965 12990 leader_election.cc:288] T 5aef9fa2349e46f8acbb93581f457580 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [CANDIDATE]: Term 1 election: Requested vote from peers ef281d65c9354fc483333fba732d64b4 (127.4.202.129:45759), 6ba18eeef9a74b97ada822d2a00cfdf3 (127.4.202.130:44939)
I20260926 06:20:54.388103 12628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5aef9fa2349e46f8acbb93581f457580" candidate_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef281d65c9354fc483333fba732d64b4"
I20260926 06:20:54.388123 12768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5aef9fa2349e46f8acbb93581f457580" candidate_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3"
I20260926 06:20:54.388203 12768 raft_consensus.cc:3037] T 5aef9fa2349e46f8acbb93581f457580 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:54.388206 12628 raft_consensus.cc:3037] T 5aef9fa2349e46f8acbb93581f457580 P ef281d65c9354fc483333fba732d64b4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:54.388815 12768 raft_consensus.cc:2454] T 5aef9fa2349e46f8acbb93581f457580 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 388fa46aa14f4d7fbfe0e887e8ec4f3c in term 1.
I20260926 06:20:54.388828 12628 raft_consensus.cc:2454] T 5aef9fa2349e46f8acbb93581f457580 P ef281d65c9354fc483333fba732d64b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 388fa46aa14f4d7fbfe0e887e8ec4f3c in term 1.
I20260926 06:20:54.388988 12842 leader_election.cc:302] T 5aef9fa2349e46f8acbb93581f457580 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 388fa46aa14f4d7fbfe0e887e8ec4f3c, 6ba18eeef9a74b97ada822d2a00cfdf3; no voters:
I20260926 06:20:54.389092 12990 raft_consensus.cc:2781] T 5aef9fa2349e46f8acbb93581f457580 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:20:54.389209 12990 raft_consensus.cc:686] T 5aef9fa2349e46f8acbb93581f457580 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 1 LEADER]: Becoming Leader. State: Replica: 388fa46aa14f4d7fbfe0e887e8ec4f3c, State: Running, Role: LEADER
I20260926 06:20:54.389264 12990 consensus_queue.cc:237] T 5aef9fa2349e46f8acbb93581f457580 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:54.389919 12470 catalog_manager.cc:5495] T 5aef9fa2349e46f8acbb93581f457580 P 388fa46aa14f4d7fbfe0e887e8ec4f3c reported cstate change: term changed from 0 to 1, leader changed from <none> to 388fa46aa14f4d7fbfe0e887e8ec4f3c (127.4.202.131). New cstate: current_term: 1 leader_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } health_report { overall_health: HEALTHY } } }
Sep 26 06:20:54 dist-test-slave-pd0t krb5kdc[12426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403653, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.190@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM
I20260926 06:20:54.461818 12520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.455272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55062 (local address 127.4.202.190:42179)
0926 06:20:54.455357 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:54.455359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:54.455432 (+ 73us) server_negotiation.cc:407] Connection header received
0926 06:20:54.455471 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:54.455471 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:54.455480 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:54.455530 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:54.455945 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.456430 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.457091 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.457270 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.457318 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.459391 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:54.459393 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:54.459394 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:54.459401 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:54.460641 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:54.461021 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:54.461022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:54.461023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:54.461081 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:54.461271 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:54.461272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:54.461272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:54.461379 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:54.461455 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:54.461599 (+ 144us) server_negotiation.cc:299] Negotiation successful
0926 06:20:54.461697 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 06:20:54.461910 13018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.455203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42179 (local address 127.0.0.1:55062)
0926 06:20:54.455354 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:54.455363 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:54.455426 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:54.455545 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:54.455546 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:54.455832 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:54.455879 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:54.455880 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.456495 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.456496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:54.457016 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:54.457017 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.457335 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.457336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:54.457374 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.457830 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:54.457834 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:54.459287 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:54.460746 (+ 1459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:54.460753 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:54.460755 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:54.460944 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:54.461148 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:54.461149 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:54.461149 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:54.461207 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:54.461445 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:54.461452 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:54.461495 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 06:20:54.461579 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 06:20:54.461754 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":83,"thread_start_us":65,"threads_started":1}
I20260926 06:20:54.463754 12470 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:55062:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 06:20:54.463979 12470 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:20:54.465449 12748 tablet_service.cc:1467] Processing CreateTablet for tablet d64a62f127394d7cba7606476120ca91 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=39cac2d730a8465a95a2cd3166b87091]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 06:20:54.465451 12608 tablet_service.cc:1467] Processing CreateTablet for tablet d64a62f127394d7cba7606476120ca91 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=39cac2d730a8465a95a2cd3166b87091]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 06:20:54.465479 12888 tablet_service.cc:1467] Processing CreateTablet for tablet d64a62f127394d7cba7606476120ca91 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=39cac2d730a8465a95a2cd3166b87091]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 06:20:54.465606 12608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d64a62f127394d7cba7606476120ca91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:54.465606 12748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d64a62f127394d7cba7606476120ca91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:54.465648 12888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d64a62f127394d7cba7606476120ca91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:54.466948 12986 tablet_bootstrap.cc:492] T d64a62f127394d7cba7606476120ca91 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Bootstrap starting.
I20260926 06:20:54.467113 12982 tablet_bootstrap.cc:492] T d64a62f127394d7cba7606476120ca91 P 6ba18eeef9a74b97ada822d2a00cfdf3: Bootstrap starting.
I20260926 06:20:54.467113 12983 tablet_bootstrap.cc:492] T d64a62f127394d7cba7606476120ca91 P ef281d65c9354fc483333fba732d64b4: Bootstrap starting.
I20260926 06:20:54.467379 12986 tablet_bootstrap.cc:654] T d64a62f127394d7cba7606476120ca91 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:54.467483 12982 tablet_bootstrap.cc:654] T d64a62f127394d7cba7606476120ca91 P 6ba18eeef9a74b97ada822d2a00cfdf3: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:54.467499 12983 tablet_bootstrap.cc:654] T d64a62f127394d7cba7606476120ca91 P ef281d65c9354fc483333fba732d64b4: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:54.468282 12983 tablet_bootstrap.cc:492] T d64a62f127394d7cba7606476120ca91 P ef281d65c9354fc483333fba732d64b4: No bootstrap required, opened a new log
I20260926 06:20:54.468282 12982 tablet_bootstrap.cc:492] T d64a62f127394d7cba7606476120ca91 P 6ba18eeef9a74b97ada822d2a00cfdf3: No bootstrap required, opened a new log
I20260926 06:20:54.468338 12983 ts_tablet_manager.cc:1397] T d64a62f127394d7cba7606476120ca91 P ef281d65c9354fc483333fba732d64b4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 06:20:54.468369 12982 ts_tablet_manager.cc:1397] T d64a62f127394d7cba7606476120ca91 P 6ba18eeef9a74b97ada822d2a00cfdf3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 06:20:54.468436 12986 tablet_bootstrap.cc:492] T d64a62f127394d7cba7606476120ca91 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: No bootstrap required, opened a new log
I20260926 06:20:54.468470 12986 ts_tablet_manager.cc:1397] T d64a62f127394d7cba7606476120ca91 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:20:54.468478 12982 raft_consensus.cc:348] T d64a62f127394d7cba7606476120ca91 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:54.468478 12983 raft_consensus.cc:348] T d64a62f127394d7cba7606476120ca91 P ef281d65c9354fc483333fba732d64b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:54.468538 12983 raft_consensus.cc:374] T d64a62f127394d7cba7606476120ca91 P ef281d65c9354fc483333fba732d64b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:54.468538 12982 raft_consensus.cc:374] T d64a62f127394d7cba7606476120ca91 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:54.468560 12982 raft_consensus.cc:729] T d64a62f127394d7cba7606476120ca91 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ba18eeef9a74b97ada822d2a00cfdf3, State: Initialized, Role: FOLLOWER
I20260926 06:20:54.468560 12983 raft_consensus.cc:729] T d64a62f127394d7cba7606476120ca91 P ef281d65c9354fc483333fba732d64b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef281d65c9354fc483333fba732d64b4, State: Initialized, Role: FOLLOWER
I20260926 06:20:54.468576 12986 raft_consensus.cc:348] T d64a62f127394d7cba7606476120ca91 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:54.468609 12983 consensus_queue.cc:260] T d64a62f127394d7cba7606476120ca91 P ef281d65c9354fc483333fba732d64b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:54.468611 12982 consensus_queue.cc:260] T d64a62f127394d7cba7606476120ca91 P 6ba18eeef9a74b97ada822d2a00cfdf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:54.468654 12986 raft_consensus.cc:374] T d64a62f127394d7cba7606476120ca91 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:54.468681 12986 raft_consensus.cc:729] T d64a62f127394d7cba7606476120ca91 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 388fa46aa14f4d7fbfe0e887e8ec4f3c, State: Initialized, Role: FOLLOWER
I20260926 06:20:54.468699 12982 ts_tablet_manager.cc:1428] T d64a62f127394d7cba7606476120ca91 P 6ba18eeef9a74b97ada822d2a00cfdf3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 06:20:54.468699 12983 ts_tablet_manager.cc:1428] T d64a62f127394d7cba7606476120ca91 P ef281d65c9354fc483333fba732d64b4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:54.468724 12986 consensus_queue.cc:260] T d64a62f127394d7cba7606476120ca91 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:54.468816 12986 ts_tablet_manager.cc:1428] T d64a62f127394d7cba7606476120ca91 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:54.468847 12990 tablet_replica.cc:439] T d64a62f127394d7cba7606476120ca91 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } } }
I20260926 06:20:54.468817 12989 tablet_replica.cc:439] T d64a62f127394d7cba7606476120ca91 P ef281d65c9354fc483333fba732d64b4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } } }
I20260926 06:20:54.468909 12990 tablet_replica.cc:442] T d64a62f127394d7cba7606476120ca91 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:20:54.468910 12989 tablet_replica.cc:442] T d64a62f127394d7cba7606476120ca91 P ef281d65c9354fc483333fba732d64b4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:20:54.469013 12992 tablet_replica.cc:439] T d64a62f127394d7cba7606476120ca91 P 6ba18eeef9a74b97ada822d2a00cfdf3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } } }
I20260926 06:20:54.469091 12992 tablet_replica.cc:442] T d64a62f127394d7cba7606476120ca91 P 6ba18eeef9a74b97ada822d2a00cfdf3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:20:54.686420 12989 raft_consensus.cc:482] T d64a62f127394d7cba7606476120ca91 P ef281d65c9354fc483333fba732d64b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:20:54.686509 12989 raft_consensus.cc:504] T d64a62f127394d7cba7606476120ca91 P ef281d65c9354fc483333fba732d64b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:54.686910 12989 leader_election.cc:288] T d64a62f127394d7cba7606476120ca91 P ef281d65c9354fc483333fba732d64b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ba18eeef9a74b97ada822d2a00cfdf3 (127.4.202.130:44939), 388fa46aa14f4d7fbfe0e887e8ec4f3c (127.4.202.131:35099)
I20260926 06:20:54.689931 12981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.687174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:54637 (local address 127.4.202.130:44939)
0926 06:20:54.687248 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:54.687251 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:54.687349 (+ 98us) server_negotiation.cc:407] Connection header received
0926 06:20:54.687447 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:54.687448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:54.687458 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:54.687514 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:20:54.687684 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.688183 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.689211 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.689566 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.689661 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.689669 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:54.689834 (+ 165us) server_negotiation.cc:299] Negotiation successful
0926 06:20:54.689873 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 06:20:54.689949 13022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.687105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:44939 (local address 127.4.202.129:54637)
0926 06:20:54.687258 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:54.687270 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:54.687369 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:54.687523 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:54.687524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:54.687526 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:20:54.687598 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:54.687600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.688340 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.688341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:54.689117 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:54.689119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.689662 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.689663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:54.689737 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.689740 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:20:54.689806 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 06:20:54.689854 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":31,"thread_start_us":64,"threads_started":1}
I20260926 06:20:54.690058 13024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.687396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:40551 (local address 127.4.202.131:35099)
0926 06:20:54.687487 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:54.687489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:54.687498 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:20:54.687571 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:54.687572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:54.687582 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:54.687657 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:20:54.687841 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.688314 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.689319 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.689702 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.689797 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.689804 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:54.689952 (+ 148us) server_negotiation.cc:299] Negotiation successful
0926 06:20:54.689993 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74,"thread_start_us":41,"threads_started":1}
I20260926 06:20:54.690068 13023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.687336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:35099 (local address 127.4.202.129:40551)
0926 06:20:54.687441 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:54.687451 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:54.687505 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:54.687675 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:54.687675 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:54.687677 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:20:54.687764 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:54.687765 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.688475 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.688476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:54.689225 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:54.689227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.689793 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.689794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:54.689866 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.689870 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:20:54.689945 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 06:20:54.689983 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":40,"threads_started":1}
I20260926 06:20:54.690392 12908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d64a62f127394d7cba7606476120ca91" candidate_uuid: "ef281d65c9354fc483333fba732d64b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" is_pre_election: true
I20260926 06:20:54.690428 12768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d64a62f127394d7cba7606476120ca91" candidate_uuid: "ef281d65c9354fc483333fba732d64b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" is_pre_election: true
I20260926 06:20:54.690485 12908 raft_consensus.cc:2454] T d64a62f127394d7cba7606476120ca91 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef281d65c9354fc483333fba732d64b4 in term 0.
I20260926 06:20:54.690518 12768 raft_consensus.cc:2454] T d64a62f127394d7cba7606476120ca91 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef281d65c9354fc483333fba732d64b4 in term 0.
I20260926 06:20:54.690686 12562 leader_election.cc:302] T d64a62f127394d7cba7606476120ca91 P ef281d65c9354fc483333fba732d64b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 388fa46aa14f4d7fbfe0e887e8ec4f3c, ef281d65c9354fc483333fba732d64b4; no voters:
I20260926 06:20:54.690865 12989 raft_consensus.cc:2781] T d64a62f127394d7cba7606476120ca91 P ef281d65c9354fc483333fba732d64b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:20:54.690922 12989 raft_consensus.cc:482] T d64a62f127394d7cba7606476120ca91 P ef281d65c9354fc483333fba732d64b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:20:54.690958 12989 raft_consensus.cc:3037] T d64a62f127394d7cba7606476120ca91 P ef281d65c9354fc483333fba732d64b4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:54.691599 12989 raft_consensus.cc:504] T d64a62f127394d7cba7606476120ca91 P ef281d65c9354fc483333fba732d64b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:54.691726 12989 leader_election.cc:288] T d64a62f127394d7cba7606476120ca91 P ef281d65c9354fc483333fba732d64b4 [CANDIDATE]: Term 1 election: Requested vote from peers 6ba18eeef9a74b97ada822d2a00cfdf3 (127.4.202.130:44939), 388fa46aa14f4d7fbfe0e887e8ec4f3c (127.4.202.131:35099)
I20260926 06:20:54.691879 12768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d64a62f127394d7cba7606476120ca91" candidate_uuid: "ef281d65c9354fc483333fba732d64b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3"
I20260926 06:20:54.691908 12908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d64a62f127394d7cba7606476120ca91" candidate_uuid: "ef281d65c9354fc483333fba732d64b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c"
I20260926 06:20:54.691951 12768 raft_consensus.cc:3037] T d64a62f127394d7cba7606476120ca91 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:54.691980 12908 raft_consensus.cc:3037] T d64a62f127394d7cba7606476120ca91 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:54.692586 12768 raft_consensus.cc:2454] T d64a62f127394d7cba7606476120ca91 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef281d65c9354fc483333fba732d64b4 in term 1.
I20260926 06:20:54.692639 12908 raft_consensus.cc:2454] T d64a62f127394d7cba7606476120ca91 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef281d65c9354fc483333fba732d64b4 in term 1.
I20260926 06:20:54.692773 12562 leader_election.cc:302] T d64a62f127394d7cba7606476120ca91 P ef281d65c9354fc483333fba732d64b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ba18eeef9a74b97ada822d2a00cfdf3, ef281d65c9354fc483333fba732d64b4; no voters:
I20260926 06:20:54.692879 12989 raft_consensus.cc:2781] T d64a62f127394d7cba7606476120ca91 P ef281d65c9354fc483333fba732d64b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:20:54.693001 12989 raft_consensus.cc:686] T d64a62f127394d7cba7606476120ca91 P ef281d65c9354fc483333fba732d64b4 [term 1 LEADER]: Becoming Leader. State: Replica: ef281d65c9354fc483333fba732d64b4, State: Running, Role: LEADER
I20260926 06:20:54.693063 12989 consensus_queue.cc:237] T d64a62f127394d7cba7606476120ca91 P ef281d65c9354fc483333fba732d64b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:54.693444 13025 tablet_replica.cc:439] T d64a62f127394d7cba7606476120ca91 P ef281d65c9354fc483333fba732d64b4: TxnStatusTablet state changed. Reason: New leader ef281d65c9354fc483333fba732d64b4. Latest consensus state: current_term: 1 leader_uuid: "ef281d65c9354fc483333fba732d64b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } } }
I20260926 06:20:54.693549 13025 tablet_replica.cc:442] T d64a62f127394d7cba7606476120ca91 P ef281d65c9354fc483333fba732d64b4: This TxnStatusTablet replica's current role is: LEADER
I20260926 06:20:54.693722 13027 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 06:20:54.693836 13027 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 06:20:54.693848 12470 catalog_manager.cc:5495] T d64a62f127394d7cba7606476120ca91 P ef281d65c9354fc483333fba732d64b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef281d65c9354fc483333fba732d64b4 (127.4.202.129). New cstate: current_term: 1 leader_uuid: "ef281d65c9354fc483333fba732d64b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } health_report { overall_health: UNKNOWN } } }
I20260926 06:20:54.751624 13018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.748933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:45759 (local address 127.0.0.1:47598)
0926 06:20:54.748990 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:54.749001 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:54.749069 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:54.749256 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:54.749257 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:54.749258 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 06:20:54.749303 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:54.749305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.750145 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.750145 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:54.750743 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:54.750745 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.751176 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.751176 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:54.751222 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.751235 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 06:20:54.751494 (+ 259us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 06:20:54.751499 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 06:20:54.751533 (+ 34us) client_negotiation.cc:240] Negotiation successful
0926 06:20:54.751570 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260926 06:20:54.751633 12979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.748973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47598 (local address 127.4.202.129:45759)
0926 06:20:54.749018 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:54.749020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:54.749060 (+ 40us) server_negotiation.cc:407] Connection header received
0926 06:20:54.749155 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:54.749156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:54.749174 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:54.749253 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:20:54.749347 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.750065 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.750818 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.751098 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.751169 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.751320 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:20:54.751430 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:20:54.751498 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:54.751542 (+ 44us) server_negotiation.cc:299] Negotiation successful
0926 06:20:54.751560 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 06:20:54.752856 12908 raft_consensus.cc:1261] T d64a62f127394d7cba7606476120ca91 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 1 FOLLOWER]: Refusing update from remote peer ef281d65c9354fc483333fba732d64b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:20:54.752959 12768 raft_consensus.cc:1261] T d64a62f127394d7cba7606476120ca91 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 1 FOLLOWER]: Refusing update from remote peer ef281d65c9354fc483333fba732d64b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:20:54.753113 12989 consensus_queue.cc:1035] T d64a62f127394d7cba7606476120ca91 P ef281d65c9354fc483333fba732d64b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:20:54.753219 13025 consensus_queue.cc:1035] T d64a62f127394d7cba7606476120ca91 P ef281d65c9354fc483333fba732d64b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:20:54.753798 12990 tablet_replica.cc:439] T d64a62f127394d7cba7606476120ca91 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: TxnStatusTablet state changed. Reason: New leader ef281d65c9354fc483333fba732d64b4. Latest consensus state: current_term: 1 leader_uuid: "ef281d65c9354fc483333fba732d64b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } } }
I20260926 06:20:54.753894 12990 tablet_replica.cc:442] T d64a62f127394d7cba7606476120ca91 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:20:54.754416 12992 tablet_replica.cc:439] T d64a62f127394d7cba7606476120ca91 P 6ba18eeef9a74b97ada822d2a00cfdf3: TxnStatusTablet state changed. Reason: New leader ef281d65c9354fc483333fba732d64b4. Latest consensus state: current_term: 1 leader_uuid: "ef281d65c9354fc483333fba732d64b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } } }
I20260926 06:20:54.754498 12992 tablet_replica.cc:442] T d64a62f127394d7cba7606476120ca91 P 6ba18eeef9a74b97ada822d2a00cfdf3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:20:54.754993 13025 tablet_replica.cc:439] T d64a62f127394d7cba7606476120ca91 P ef281d65c9354fc483333fba732d64b4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ef281d65c9354fc483333fba732d64b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } } }
I20260926 06:20:54.755024 12989 tablet_replica.cc:439] T d64a62f127394d7cba7606476120ca91 P ef281d65c9354fc483333fba732d64b4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ef281d65c9354fc483333fba732d64b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } } }
I20260926 06:20:54.755079 13025 tablet_replica.cc:442] T d64a62f127394d7cba7606476120ca91 P ef281d65c9354fc483333fba732d64b4: This TxnStatusTablet replica's current role is: LEADER
I20260926 06:20:54.755092 12989 tablet_replica.cc:442] T d64a62f127394d7cba7606476120ca91 P ef281d65c9354fc483333fba732d64b4: This TxnStatusTablet replica's current role is: LEADER
I20260926 06:20:54.755286 12990 tablet_replica.cc:439] T d64a62f127394d7cba7606476120ca91 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ef281d65c9354fc483333fba732d64b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } } }
I20260926 06:20:54.755345 13030 mvcc.cc:205] Tried to move back new op lower bound from 7333493369865826304 to 7333493369623326720. Current Snapshot: MvccSnapshot[applied={T|T < 7333493369865826304}]
I20260926 06:20:54.755378 12990 tablet_replica.cc:442] T d64a62f127394d7cba7606476120ca91 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:20:54.755606 12992 tablet_replica.cc:439] T d64a62f127394d7cba7606476120ca91 P 6ba18eeef9a74b97ada822d2a00cfdf3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ef281d65c9354fc483333fba732d64b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } } }
I20260926 06:20:54.755685 12992 tablet_replica.cc:442] T d64a62f127394d7cba7606476120ca91 P 6ba18eeef9a74b97ada822d2a00cfdf3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 06:20:54.755836 13033 mvcc.cc:205] Tried to move back new op lower bound from 7333493369865826304 to 7333493369623326720. Current Snapshot: MvccSnapshot[applied={T|T < 7333493369865826304 or (T in {7333493369865826304})}]
I20260926 06:20:54.760339 13024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.757714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35298 (local address 127.4.202.131:35099)
0926 06:20:54.757753 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:54.757754 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:54.757904 (+ 150us) server_negotiation.cc:407] Connection header received
0926 06:20:54.757944 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:54.757945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:54.757966 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:54.758031 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:20:54.758133 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.758801 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.759504 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.759797 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.759887 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.760042 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:20:54.760152 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:20:54.760229 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:54.760256 (+ 27us) server_negotiation.cc:299] Negotiation successful
0926 06:20:54.760278 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 06:20:54.764945 12979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.762526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:33947 (local address 127.4.202.129:45759)
0926 06:20:54.762573 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:54.762575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:54.762697 (+ 122us) server_negotiation.cc:407] Connection header received
0926 06:20:54.762767 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:54.762768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:54.762778 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:54.762848 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:20:54.762984 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.763408 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.764175 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.764385 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.764459 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.764680 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:20:54.764739 (+ 59us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:20:54.764795 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:54.764850 (+ 55us) server_negotiation.cc:299] Negotiation successful
0926 06:20:54.764866 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 06:20:54.764954 13043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.762459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:45759 (local address 127.4.202.131:33947)
0926 06:20:54.762623 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:54.762633 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:54.762711 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:54.762851 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:54.762852 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:54.762854 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 06:20:54.762905 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:54.762906 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.763502 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.763503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:54.764098 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:54.764099 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.764491 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.764492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:54.764554 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.764564 (+ 10us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 06:20:54.764797 (+ 233us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 06:20:54.764799 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:20:54.764846 (+ 47us) client_negotiation.cc:240] Negotiation successful
0926 06:20:54.764885 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":83,"threads_started":1}
I20260926 06:20:54.767341 12768 raft_consensus.cc:1261] T 5aef9fa2349e46f8acbb93581f457580 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 1 FOLLOWER]: Refusing update from remote peer 388fa46aa14f4d7fbfe0e887e8ec4f3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:20:54.767349 12628 raft_consensus.cc:1261] T 5aef9fa2349e46f8acbb93581f457580 P ef281d65c9354fc483333fba732d64b4 [term 1 FOLLOWER]: Refusing update from remote peer 388fa46aa14f4d7fbfe0e887e8ec4f3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:20:54.767627 13011 consensus_queue.cc:1035] T 5aef9fa2349e46f8acbb93581f457580 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:20:54.767778 12990 consensus_queue.cc:1035] T 5aef9fa2349e46f8acbb93581f457580 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:20:54.776623 13024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.773603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:54997 (local address 127.4.202.131:35099)
0926 06:20:54.773641 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:54.773642 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:54.773818 (+ 176us) server_negotiation.cc:407] Connection header received
0926 06:20:54.773840 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:54.773841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:54.773850 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:54.773946 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:20:54.774081 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.774756 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.775618 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.775915 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.776027 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.776289 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:20:54.776357 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:20:54.776453 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:54.776534 (+ 81us) server_negotiation.cc:299] Negotiation successful
0926 06:20:54.776554 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 06:20:54.776691 13047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.773540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:35099 (local address 127.4.202.129:54997)
0926 06:20:54.773674 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:54.773685 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:54.773741 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:54.773953 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:54.773954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:54.773955 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 06:20:54.774009 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:54.774010 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.774824 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.774825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:54.775522 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:54.775525 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.775998 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.775999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:54.776108 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.776139 (+ 31us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 06:20:54.776458 (+ 319us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 06:20:54.776461 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:20:54.776514 (+ 53us) client_negotiation.cc:240] Negotiation successful
0926 06:20:54.776561 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":78,"threads_started":1}
I20260926 06:20:54.783572 12471 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:20:54.783658 12471 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:20:54.784329 12471 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b4c397f387024f17b7d62d585e95e5cf: Sending DeleteTablet for 3 replicas of tablet 5aef9fa2349e46f8acbb93581f457580
I20260926 06:20:54.784569 12608 tablet_service.cc:1514] Processing DeleteTablet for tablet 5aef9fa2349e46f8acbb93581f457580 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:20:54 UTC) from {username='oryx', principal='oryx/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:47408
I20260926 06:20:54.784575 12888 tablet_service.cc:1514] Processing DeleteTablet for tablet 5aef9fa2349e46f8acbb93581f457580 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:20:54 UTC) from {username='oryx', principal='oryx/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:57826
I20260926 06:20:54.784783 13051 tablet_replica.cc:330] T 5aef9fa2349e46f8acbb93581f457580 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: stopping tablet replica
I20260926 06:20:54.784780 12748 tablet_service.cc:1514] Processing DeleteTablet for tablet 5aef9fa2349e46f8acbb93581f457580 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:20:54 UTC) from {username='oryx', principal='oryx/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:58112
I20260926 06:20:54.784842 13051 raft_consensus.cc:2229] T 5aef9fa2349e46f8acbb93581f457580 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:20:54.784931 13051 raft_consensus.cc:2258] T 5aef9fa2349e46f8acbb93581f457580 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:20:54.785175 13051 ts_tablet_manager.cc:1905] T 5aef9fa2349e46f8acbb93581f457580 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:20:54.786852 13052 tablet_replica.cc:330] T 5aef9fa2349e46f8acbb93581f457580 P ef281d65c9354fc483333fba732d64b4: stopping tablet replica
I20260926 06:20:54.786942 13052 raft_consensus.cc:2229] T 5aef9fa2349e46f8acbb93581f457580 P ef281d65c9354fc483333fba732d64b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:20:54.786978 13052 raft_consensus.cc:2258] T 5aef9fa2349e46f8acbb93581f457580 P ef281d65c9354fc483333fba732d64b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:20:54.787101 13051 ts_tablet_manager.cc:1918] T 5aef9fa2349e46f8acbb93581f457580 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 06:20:54.787147 13051 log.cc:1196] T 5aef9fa2349e46f8acbb93581f457580 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-2/wal/wals/5aef9fa2349e46f8acbb93581f457580
I20260926 06:20:54.787388 13051 ts_tablet_manager.cc:1939] T 5aef9fa2349e46f8acbb93581f457580 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Deleting consensus metadata
I20260926 06:20:54.787662 12458 catalog_manager.cc:4841] TS 388fa46aa14f4d7fbfe0e887e8ec4f3c (127.4.202.131:35099): tablet 5aef9fa2349e46f8acbb93581f457580 (table test-table [id=643cb8a5f16f41198a258dd8b949f1d7]) successfully deleted
I20260926 06:20:54.787951 13052 ts_tablet_manager.cc:1905] T 5aef9fa2349e46f8acbb93581f457580 P ef281d65c9354fc483333fba732d64b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:20:54.789510 13052 ts_tablet_manager.cc:1918] T 5aef9fa2349e46f8acbb93581f457580 P ef281d65c9354fc483333fba732d64b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 06:20:54.789568 13052 log.cc:1196] T 5aef9fa2349e46f8acbb93581f457580 P ef281d65c9354fc483333fba732d64b4: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-0/wal/wals/5aef9fa2349e46f8acbb93581f457580
I20260926 06:20:54.789790 13052 ts_tablet_manager.cc:1939] T 5aef9fa2349e46f8acbb93581f457580 P ef281d65c9354fc483333fba732d64b4: Deleting consensus metadata
I20260926 06:20:54.790105 12456 catalog_manager.cc:4841] TS ef281d65c9354fc483333fba732d64b4 (127.4.202.129:45759): tablet 5aef9fa2349e46f8acbb93581f457580 (table test-table [id=643cb8a5f16f41198a258dd8b949f1d7]) successfully deleted
I20260926 06:20:54.790670 13053 tablet_replica.cc:330] T 5aef9fa2349e46f8acbb93581f457580 P 6ba18eeef9a74b97ada822d2a00cfdf3: stopping tablet replica
I20260926 06:20:54.790742 13053 raft_consensus.cc:2229] T 5aef9fa2349e46f8acbb93581f457580 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:20:54.790776 13053 raft_consensus.cc:2258] T 5aef9fa2349e46f8acbb93581f457580 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:20:54.791021 13053 ts_tablet_manager.cc:1905] T 5aef9fa2349e46f8acbb93581f457580 P 6ba18eeef9a74b97ada822d2a00cfdf3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:20:54.792685 13053 ts_tablet_manager.cc:1918] T 5aef9fa2349e46f8acbb93581f457580 P 6ba18eeef9a74b97ada822d2a00cfdf3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 06:20:54.792752 13053 log.cc:1196] T 5aef9fa2349e46f8acbb93581f457580 P 6ba18eeef9a74b97ada822d2a00cfdf3: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-1/wal/wals/5aef9fa2349e46f8acbb93581f457580
I20260926 06:20:54.792953 13053 ts_tablet_manager.cc:1939] T 5aef9fa2349e46f8acbb93581f457580 P 6ba18eeef9a74b97ada822d2a00cfdf3: Deleting consensus metadata
I20260926 06:20:54.793351 12458 catalog_manager.cc:4841] TS 6ba18eeef9a74b97ada822d2a00cfdf3 (127.4.202.130:44939): tablet 5aef9fa2349e46f8acbb93581f457580 (table test-table [id=643cb8a5f16f41198a258dd8b949f1d7]) successfully deleted
Sep 26 06:20:54 dist-test-slave-pd0t krb5kdc[12426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 06:20:54 dist-test-slave-pd0t krb5kdc[12426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:20:54 dist-test-slave-pd0t krb5kdc[12426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
W20260926 06:20:54.819680 12520 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:20:54.812717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55070 (local address 127.4.202.190:42179)
0926 06:20:54.812766 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:54.812768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:54.812873 (+ 105us) server_negotiation.cc:407] Connection header received
0926 06:20:54.812920 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:54.812921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:54.812933 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:54.813002 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:54.813768 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.814536 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.815686 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.816006 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.816081 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.819619 (+ 3538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55070: BlockingRecv error: recv got EOF from 127.0.0.1:55070 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Sep 26 06:20:54 dist-test-slave-pd0t krb5kdc[12426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM
I20260926 06:20:54.829137 12520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:54.820676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55086 (local address 127.4.202.190:42179)
0926 06:20:54.820709 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:54.820711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:54.820775 (+ 64us) server_negotiation.cc:407] Connection header received
0926 06:20:54.820821 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:54.820822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:54.820830 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:54.820889 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:54.821426 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.822165 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.823219 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:54.823522 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:54.823593 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:54.826096 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:54.826097 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:54.826097 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:54.826104 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:54.827786 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:54.828217 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:54.828219 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:54.828219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:54.828281 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:54.828507 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:54.828508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:54.828508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:54.828643 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:54.828747 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:54.828878 (+ 131us) server_negotiation.cc:299] Negotiation successful
0926 06:20:54.828950 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 06:20:54.830704 12471 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55086:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:20:54.830881 12471 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:20:54.832284 12748 tablet_service.cc:1467] Processing CreateTablet for tablet 81bc9f0c33474340b76ea5263795ee9f (DEFAULT_TABLE table=test-table [id=4fcecbf53fa34fadba94b5a2bd4c8a87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:20:54.832343 12608 tablet_service.cc:1467] Processing CreateTablet for tablet 81bc9f0c33474340b76ea5263795ee9f (DEFAULT_TABLE table=test-table [id=4fcecbf53fa34fadba94b5a2bd4c8a87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:20:54.832412 12748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81bc9f0c33474340b76ea5263795ee9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:54.832459 12608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81bc9f0c33474340b76ea5263795ee9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:54.833734 12888 tablet_service.cc:1467] Processing CreateTablet for tablet 81bc9f0c33474340b76ea5263795ee9f (DEFAULT_TABLE table=test-table [id=4fcecbf53fa34fadba94b5a2bd4c8a87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:20:54.833842 12888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81bc9f0c33474340b76ea5263795ee9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:54.835165 12986 tablet_bootstrap.cc:492] T 81bc9f0c33474340b76ea5263795ee9f P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Bootstrap starting.
I20260926 06:20:54.835176 12983 tablet_bootstrap.cc:492] T 81bc9f0c33474340b76ea5263795ee9f P ef281d65c9354fc483333fba732d64b4: Bootstrap starting.
I20260926 06:20:54.835546 12983 tablet_bootstrap.cc:654] T 81bc9f0c33474340b76ea5263795ee9f P ef281d65c9354fc483333fba732d64b4: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:54.835604 12986 tablet_bootstrap.cc:654] T 81bc9f0c33474340b76ea5263795ee9f P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:54.835836 12982 tablet_bootstrap.cc:492] T 81bc9f0c33474340b76ea5263795ee9f P 6ba18eeef9a74b97ada822d2a00cfdf3: Bootstrap starting.
I20260926 06:20:54.836127 12982 tablet_bootstrap.cc:654] T 81bc9f0c33474340b76ea5263795ee9f P 6ba18eeef9a74b97ada822d2a00cfdf3: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:54.836355 12986 tablet_bootstrap.cc:492] T 81bc9f0c33474340b76ea5263795ee9f P 388fa46aa14f4d7fbfe0e887e8ec4f3c: No bootstrap required, opened a new log
I20260926 06:20:54.836402 12986 ts_tablet_manager.cc:1397] T 81bc9f0c33474340b76ea5263795ee9f P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260926 06:20:54.836549 12986 raft_consensus.cc:348] T 81bc9f0c33474340b76ea5263795ee9f P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:54.836611 12986 raft_consensus.cc:374] T 81bc9f0c33474340b76ea5263795ee9f P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:54.836630 12986 raft_consensus.cc:729] T 81bc9f0c33474340b76ea5263795ee9f P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 388fa46aa14f4d7fbfe0e887e8ec4f3c, State: Initialized, Role: FOLLOWER
I20260926 06:20:54.836670 12986 consensus_queue.cc:260] T 81bc9f0c33474340b76ea5263795ee9f P 388fa46aa14f4d7fbfe0e887e8ec4f3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:54.836764 12986 ts_tablet_manager.cc:1428] T 81bc9f0c33474340b76ea5263795ee9f P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:54.837086 12982 tablet_bootstrap.cc:492] T 81bc9f0c33474340b76ea5263795ee9f P 6ba18eeef9a74b97ada822d2a00cfdf3: No bootstrap required, opened a new log
I20260926 06:20:54.837126 12982 ts_tablet_manager.cc:1397] T 81bc9f0c33474340b76ea5263795ee9f P 6ba18eeef9a74b97ada822d2a00cfdf3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 06:20:54.837245 12982 raft_consensus.cc:348] T 81bc9f0c33474340b76ea5263795ee9f P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:54.837308 12982 raft_consensus.cc:374] T 81bc9f0c33474340b76ea5263795ee9f P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:54.837325 12982 raft_consensus.cc:729] T 81bc9f0c33474340b76ea5263795ee9f P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ba18eeef9a74b97ada822d2a00cfdf3, State: Initialized, Role: FOLLOWER
I20260926 06:20:54.837358 12982 consensus_queue.cc:260] T 81bc9f0c33474340b76ea5263795ee9f P 6ba18eeef9a74b97ada822d2a00cfdf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:54.837430 12982 ts_tablet_manager.cc:1428] T 81bc9f0c33474340b76ea5263795ee9f P 6ba18eeef9a74b97ada822d2a00cfdf3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:54.837728 12983 tablet_bootstrap.cc:492] T 81bc9f0c33474340b76ea5263795ee9f P ef281d65c9354fc483333fba732d64b4: No bootstrap required, opened a new log
I20260926 06:20:54.837765 12983 ts_tablet_manager.cc:1397] T 81bc9f0c33474340b76ea5263795ee9f P ef281d65c9354fc483333fba732d64b4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260926 06:20:54.837903 12983 raft_consensus.cc:348] T 81bc9f0c33474340b76ea5263795ee9f P ef281d65c9354fc483333fba732d64b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:54.837963 12983 raft_consensus.cc:374] T 81bc9f0c33474340b76ea5263795ee9f P ef281d65c9354fc483333fba732d64b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:54.837981 12983 raft_consensus.cc:729] T 81bc9f0c33474340b76ea5263795ee9f P ef281d65c9354fc483333fba732d64b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef281d65c9354fc483333fba732d64b4, State: Initialized, Role: FOLLOWER
I20260926 06:20:54.838018 12983 consensus_queue.cc:260] T 81bc9f0c33474340b76ea5263795ee9f P ef281d65c9354fc483333fba732d64b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:54.838086 12983 ts_tablet_manager.cc:1428] T 81bc9f0c33474340b76ea5263795ee9f P ef281d65c9354fc483333fba732d64b4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:54.983510 13011 raft_consensus.cc:482] T 81bc9f0c33474340b76ea5263795ee9f P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:20:54.983604 13011 raft_consensus.cc:504] T 81bc9f0c33474340b76ea5263795ee9f P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:54.983788 13011 leader_election.cc:288] T 81bc9f0c33474340b76ea5263795ee9f P 388fa46aa14f4d7fbfe0e887e8ec4f3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef281d65c9354fc483333fba732d64b4 (127.4.202.129:45759), 6ba18eeef9a74b97ada822d2a00cfdf3 (127.4.202.130:44939)
I20260926 06:20:54.983989 12768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81bc9f0c33474340b76ea5263795ee9f" candidate_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" is_pre_election: true
I20260926 06:20:54.983989 12628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81bc9f0c33474340b76ea5263795ee9f" candidate_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef281d65c9354fc483333fba732d64b4" is_pre_election: true
I20260926 06:20:54.984088 12628 raft_consensus.cc:2454] T 81bc9f0c33474340b76ea5263795ee9f P ef281d65c9354fc483333fba732d64b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 388fa46aa14f4d7fbfe0e887e8ec4f3c in term 0.
I20260926 06:20:54.984088 12768 raft_consensus.cc:2454] T 81bc9f0c33474340b76ea5263795ee9f P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 388fa46aa14f4d7fbfe0e887e8ec4f3c in term 0.
I20260926 06:20:54.984266 12842 leader_election.cc:302] T 81bc9f0c33474340b76ea5263795ee9f P 388fa46aa14f4d7fbfe0e887e8ec4f3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 388fa46aa14f4d7fbfe0e887e8ec4f3c, 6ba18eeef9a74b97ada822d2a00cfdf3; no voters:
I20260926 06:20:54.984380 13011 raft_consensus.cc:2781] T 81bc9f0c33474340b76ea5263795ee9f P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:20:54.984421 13011 raft_consensus.cc:482] T 81bc9f0c33474340b76ea5263795ee9f P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:20:54.984440 13011 raft_consensus.cc:3037] T 81bc9f0c33474340b76ea5263795ee9f P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:54.985081 13011 raft_consensus.cc:504] T 81bc9f0c33474340b76ea5263795ee9f P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:54.985224 13011 leader_election.cc:288] T 81bc9f0c33474340b76ea5263795ee9f P 388fa46aa14f4d7fbfe0e887e8ec4f3c [CANDIDATE]: Term 1 election: Requested vote from peers ef281d65c9354fc483333fba732d64b4 (127.4.202.129:45759), 6ba18eeef9a74b97ada822d2a00cfdf3 (127.4.202.130:44939)
I20260926 06:20:54.985359 12628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81bc9f0c33474340b76ea5263795ee9f" candidate_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef281d65c9354fc483333fba732d64b4"
I20260926 06:20:54.985389 12768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81bc9f0c33474340b76ea5263795ee9f" candidate_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3"
I20260926 06:20:54.985463 12768 raft_consensus.cc:3037] T 81bc9f0c33474340b76ea5263795ee9f P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:54.985483 12628 raft_consensus.cc:3037] T 81bc9f0c33474340b76ea5263795ee9f P ef281d65c9354fc483333fba732d64b4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:54.986125 12768 raft_consensus.cc:2454] T 81bc9f0c33474340b76ea5263795ee9f P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 388fa46aa14f4d7fbfe0e887e8ec4f3c in term 1.
I20260926 06:20:54.986138 12628 raft_consensus.cc:2454] T 81bc9f0c33474340b76ea5263795ee9f P ef281d65c9354fc483333fba732d64b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 388fa46aa14f4d7fbfe0e887e8ec4f3c in term 1.
I20260926 06:20:54.986289 12842 leader_election.cc:302] T 81bc9f0c33474340b76ea5263795ee9f P 388fa46aa14f4d7fbfe0e887e8ec4f3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 388fa46aa14f4d7fbfe0e887e8ec4f3c, 6ba18eeef9a74b97ada822d2a00cfdf3; no voters:
I20260926 06:20:54.986385 13011 raft_consensus.cc:2781] T 81bc9f0c33474340b76ea5263795ee9f P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:20:54.986470 13011 raft_consensus.cc:686] T 81bc9f0c33474340b76ea5263795ee9f P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 1 LEADER]: Becoming Leader. State: Replica: 388fa46aa14f4d7fbfe0e887e8ec4f3c, State: Running, Role: LEADER
I20260926 06:20:54.986527 13011 consensus_queue.cc:237] T 81bc9f0c33474340b76ea5263795ee9f P 388fa46aa14f4d7fbfe0e887e8ec4f3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:54.987114 12471 catalog_manager.cc:5495] T 81bc9f0c33474340b76ea5263795ee9f P 388fa46aa14f4d7fbfe0e887e8ec4f3c reported cstate change: term changed from 0 to 1, leader changed from <none> to 388fa46aa14f4d7fbfe0e887e8ec4f3c (127.4.202.131). New cstate: current_term: 1 leader_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } health_report { overall_health: HEALTHY } } }
I20260926 06:20:55.019119 13024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:55.016721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35310 (local address 127.4.202.131:35099)
0926 06:20:55.016789 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:55.016791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:55.016801 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:20:55.016850 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:55.016851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:55.016860 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:55.016931 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:20:55.017081 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:55.017586 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:55.018343 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:55.018559 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:55.018697 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:55.018862 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:20:55.018903 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:20:55.018965 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:55.019019 (+ 54us) server_negotiation.cc:299] Negotiation successful
0926 06:20:55.019038 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 06:20:55.019897 12628 raft_consensus.cc:1261] T 81bc9f0c33474340b76ea5263795ee9f P ef281d65c9354fc483333fba732d64b4 [term 1 FOLLOWER]: Refusing update from remote peer 388fa46aa14f4d7fbfe0e887e8ec4f3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:20:55.019889 12768 raft_consensus.cc:1261] T 81bc9f0c33474340b76ea5263795ee9f P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 1 FOLLOWER]: Refusing update from remote peer 388fa46aa14f4d7fbfe0e887e8ec4f3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:20:55.020088 12990 consensus_queue.cc:1035] T 81bc9f0c33474340b76ea5263795ee9f P 388fa46aa14f4d7fbfe0e887e8ec4f3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:20:55.020208 13011 consensus_queue.cc:1035] T 81bc9f0c33474340b76ea5263795ee9f P 388fa46aa14f4d7fbfe0e887e8ec4f3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:20:55.022888 12470 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:20:55.022977 12470 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:20:55.023578 12470 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b4c397f387024f17b7d62d585e95e5cf: Sending DeleteTablet for 3 replicas of tablet 81bc9f0c33474340b76ea5263795ee9f
I20260926 06:20:55.023806 12608 tablet_service.cc:1514] Processing DeleteTablet for tablet 81bc9f0c33474340b76ea5263795ee9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:20:55 UTC) from {username='oryx', principal='oryx/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:47408
I20260926 06:20:55.023831 12748 tablet_service.cc:1514] Processing DeleteTablet for tablet 81bc9f0c33474340b76ea5263795ee9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:20:55 UTC) from {username='oryx', principal='oryx/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:58112
I20260926 06:20:55.023910 12888 tablet_service.cc:1514] Processing DeleteTablet for tablet 81bc9f0c33474340b76ea5263795ee9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:20:55 UTC) from {username='oryx', principal='oryx/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:57826
I20260926 06:20:55.023913 13052 tablet_replica.cc:330] T 81bc9f0c33474340b76ea5263795ee9f P ef281d65c9354fc483333fba732d64b4: stopping tablet replica
I20260926 06:20:55.023976 13052 raft_consensus.cc:2229] T 81bc9f0c33474340b76ea5263795ee9f P ef281d65c9354fc483333fba732d64b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:20:55.023983 13051 tablet_replica.cc:330] T 81bc9f0c33474340b76ea5263795ee9f P 388fa46aa14f4d7fbfe0e887e8ec4f3c: stopping tablet replica
I20260926 06:20:55.024020 13052 raft_consensus.cc:2258] T 81bc9f0c33474340b76ea5263795ee9f P ef281d65c9354fc483333fba732d64b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:20:55.024024 13051 raft_consensus.cc:2229] T 81bc9f0c33474340b76ea5263795ee9f P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:20:55.024097 13051 raft_consensus.cc:2258] T 81bc9f0c33474340b76ea5263795ee9f P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:20:55.024097 13053 tablet_replica.cc:330] T 81bc9f0c33474340b76ea5263795ee9f P 6ba18eeef9a74b97ada822d2a00cfdf3: stopping tablet replica
I20260926 06:20:55.024148 13053 raft_consensus.cc:2229] T 81bc9f0c33474340b76ea5263795ee9f P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:20:55.024094 12470 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55086:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 06:20:55.024184 13053 raft_consensus.cc:2258] T 81bc9f0c33474340b76ea5263795ee9f P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 06:20:55.024221 12470 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:20:55.024430 13052 ts_tablet_manager.cc:1905] T 81bc9f0c33474340b76ea5263795ee9f P ef281d65c9354fc483333fba732d64b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:20:55.024483 13053 ts_tablet_manager.cc:1905] T 81bc9f0c33474340b76ea5263795ee9f P 6ba18eeef9a74b97ada822d2a00cfdf3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:20:55.026206 13051 ts_tablet_manager.cc:1905] T 81bc9f0c33474340b76ea5263795ee9f P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:20:55.026262 13053 ts_tablet_manager.cc:1918] T 81bc9f0c33474340b76ea5263795ee9f P 6ba18eeef9a74b97ada822d2a00cfdf3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:20:55.026314 13053 log.cc:1196] T 81bc9f0c33474340b76ea5263795ee9f P 6ba18eeef9a74b97ada822d2a00cfdf3: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-1/wal/wals/81bc9f0c33474340b76ea5263795ee9f
I20260926 06:20:55.026315 13052 ts_tablet_manager.cc:1918] T 81bc9f0c33474340b76ea5263795ee9f P ef281d65c9354fc483333fba732d64b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:20:55.026360 13052 log.cc:1196] T 81bc9f0c33474340b76ea5263795ee9f P ef281d65c9354fc483333fba732d64b4: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-0/wal/wals/81bc9f0c33474340b76ea5263795ee9f
I20260926 06:20:55.026551 13053 ts_tablet_manager.cc:1939] T 81bc9f0c33474340b76ea5263795ee9f P 6ba18eeef9a74b97ada822d2a00cfdf3: Deleting consensus metadata
I20260926 06:20:55.026554 13052 ts_tablet_manager.cc:1939] T 81bc9f0c33474340b76ea5263795ee9f P ef281d65c9354fc483333fba732d64b4: Deleting consensus metadata
I20260926 06:20:55.026773 12888 tablet_service.cc:1467] Processing CreateTablet for tablet 17b8f5254dec4c9ebb18ec77dce6bfd8 (DEFAULT_TABLE table=test-table [id=9626189142834af78b6741f431b8bffb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:20:55.026809 12748 tablet_service.cc:1467] Processing CreateTablet for tablet 17b8f5254dec4c9ebb18ec77dce6bfd8 (DEFAULT_TABLE table=test-table [id=9626189142834af78b6741f431b8bffb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:20:55.026873 12888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17b8f5254dec4c9ebb18ec77dce6bfd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:55.026891 12748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17b8f5254dec4c9ebb18ec77dce6bfd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:55.027639 12982 tablet_bootstrap.cc:492] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 6ba18eeef9a74b97ada822d2a00cfdf3: Bootstrap starting.
I20260926 06:20:55.027699 13051 ts_tablet_manager.cc:1918] T 81bc9f0c33474340b76ea5263795ee9f P 388fa46aa14f4d7fbfe0e887e8ec4f3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:20:55.027743 13051 log.cc:1196] T 81bc9f0c33474340b76ea5263795ee9f P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-2/wal/wals/81bc9f0c33474340b76ea5263795ee9f
I20260926 06:20:55.027772 12608 tablet_service.cc:1467] Processing CreateTablet for tablet 17b8f5254dec4c9ebb18ec77dce6bfd8 (DEFAULT_TABLE table=test-table [id=9626189142834af78b6741f431b8bffb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:20:55.027853 12608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17b8f5254dec4c9ebb18ec77dce6bfd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:55.027920 12982 tablet_bootstrap.cc:654] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 6ba18eeef9a74b97ada822d2a00cfdf3: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:55.027947 13051 ts_tablet_manager.cc:1939] T 81bc9f0c33474340b76ea5263795ee9f P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Deleting consensus metadata
I20260926 06:20:55.028146 12986 tablet_bootstrap.cc:492] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Bootstrap starting.
I20260926 06:20:55.028384 12458 catalog_manager.cc:4841] TS 388fa46aa14f4d7fbfe0e887e8ec4f3c (127.4.202.131:35099): tablet 81bc9f0c33474340b76ea5263795ee9f (table test-table [id=4fcecbf53fa34fadba94b5a2bd4c8a87]) successfully deleted
I20260926 06:20:55.028424 12986 tablet_bootstrap.cc:654] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:55.028923 12986 tablet_bootstrap.cc:492] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: No bootstrap required, opened a new log
I20260926 06:20:55.028990 12986 ts_tablet_manager.cc:1397] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260926 06:20:55.029072 12983 tablet_bootstrap.cc:492] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P ef281d65c9354fc483333fba732d64b4: Bootstrap starting.
I20260926 06:20:55.029095 12986 raft_consensus.cc:348] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:55.029175 12986 raft_consensus.cc:374] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:55.029201 12986 raft_consensus.cc:729] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 388fa46aa14f4d7fbfe0e887e8ec4f3c, State: Initialized, Role: FOLLOWER
I20260926 06:20:55.029222 12458 catalog_manager.cc:4841] TS 6ba18eeef9a74b97ada822d2a00cfdf3 (127.4.202.130:44939): tablet 81bc9f0c33474340b76ea5263795ee9f (table test-table [id=4fcecbf53fa34fadba94b5a2bd4c8a87]) successfully deleted
I20260926 06:20:55.029239 12986 consensus_queue.cc:260] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:55.029306 12456 catalog_manager.cc:4841] TS ef281d65c9354fc483333fba732d64b4 (127.4.202.129:45759): tablet 81bc9f0c33474340b76ea5263795ee9f (table test-table [id=4fcecbf53fa34fadba94b5a2bd4c8a87]) successfully deleted
I20260926 06:20:55.029325 12986 ts_tablet_manager.cc:1428] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:55.029388 12983 tablet_bootstrap.cc:654] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P ef281d65c9354fc483333fba732d64b4: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:55.029772 12982 tablet_bootstrap.cc:492] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 6ba18eeef9a74b97ada822d2a00cfdf3: No bootstrap required, opened a new log
I20260926 06:20:55.029830 12982 ts_tablet_manager.cc:1397] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 6ba18eeef9a74b97ada822d2a00cfdf3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:20:55.029893 12983 tablet_bootstrap.cc:492] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P ef281d65c9354fc483333fba732d64b4: No bootstrap required, opened a new log
I20260926 06:20:55.029942 12983 ts_tablet_manager.cc:1397] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P ef281d65c9354fc483333fba732d64b4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 06:20:55.029947 12982 raft_consensus.cc:348] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:55.030014 12982 raft_consensus.cc:374] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:55.030042 12982 raft_consensus.cc:729] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ba18eeef9a74b97ada822d2a00cfdf3, State: Initialized, Role: FOLLOWER
I20260926 06:20:55.030051 12983 raft_consensus.cc:348] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P ef281d65c9354fc483333fba732d64b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:55.030093 12982 consensus_queue.cc:260] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 6ba18eeef9a74b97ada822d2a00cfdf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:55.030119 12983 raft_consensus.cc:374] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P ef281d65c9354fc483333fba732d64b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:55.030148 12983 raft_consensus.cc:729] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P ef281d65c9354fc483333fba732d64b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef281d65c9354fc483333fba732d64b4, State: Initialized, Role: FOLLOWER
I20260926 06:20:55.030187 12982 ts_tablet_manager.cc:1428] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 6ba18eeef9a74b97ada822d2a00cfdf3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:55.030197 12983 consensus_queue.cc:260] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P ef281d65c9354fc483333fba732d64b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:55.030303 12983 ts_tablet_manager.cc:1428] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P ef281d65c9354fc483333fba732d64b4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 06:20:55.067757 12990 raft_consensus.cc:482] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:20:55.067860 12990 raft_consensus.cc:504] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:55.068017 12990 leader_election.cc:288] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef281d65c9354fc483333fba732d64b4 (127.4.202.129:45759), 6ba18eeef9a74b97ada822d2a00cfdf3 (127.4.202.130:44939)
I20260926 06:20:55.068199 12628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17b8f5254dec4c9ebb18ec77dce6bfd8" candidate_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef281d65c9354fc483333fba732d64b4" is_pre_election: true
I20260926 06:20:55.068192 12768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17b8f5254dec4c9ebb18ec77dce6bfd8" candidate_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" is_pre_election: true
I20260926 06:20:55.068282 12628 raft_consensus.cc:2454] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P ef281d65c9354fc483333fba732d64b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 388fa46aa14f4d7fbfe0e887e8ec4f3c in term 0.
I20260926 06:20:55.068282 12768 raft_consensus.cc:2454] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 388fa46aa14f4d7fbfe0e887e8ec4f3c in term 0.
I20260926 06:20:55.068424 12842 leader_election.cc:302] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 388fa46aa14f4d7fbfe0e887e8ec4f3c, 6ba18eeef9a74b97ada822d2a00cfdf3; no voters:
I20260926 06:20:55.068537 12990 raft_consensus.cc:2781] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:20:55.068579 12990 raft_consensus.cc:482] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:20:55.068603 12990 raft_consensus.cc:3037] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:55.069170 12990 raft_consensus.cc:504] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:55.069308 12990 leader_election.cc:288] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [CANDIDATE]: Term 1 election: Requested vote from peers ef281d65c9354fc483333fba732d64b4 (127.4.202.129:45759), 6ba18eeef9a74b97ada822d2a00cfdf3 (127.4.202.130:44939)
I20260926 06:20:55.069432 12768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17b8f5254dec4c9ebb18ec77dce6bfd8" candidate_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3"
I20260926 06:20:55.069427 12628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17b8f5254dec4c9ebb18ec77dce6bfd8" candidate_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef281d65c9354fc483333fba732d64b4"
I20260926 06:20:55.069502 12768 raft_consensus.cc:3037] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:55.069519 12628 raft_consensus.cc:3037] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P ef281d65c9354fc483333fba732d64b4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:55.070011 12768 raft_consensus.cc:2454] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 388fa46aa14f4d7fbfe0e887e8ec4f3c in term 1.
I20260926 06:20:55.070088 12628 raft_consensus.cc:2454] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P ef281d65c9354fc483333fba732d64b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 388fa46aa14f4d7fbfe0e887e8ec4f3c in term 1.
I20260926 06:20:55.070199 12842 leader_election.cc:302] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 388fa46aa14f4d7fbfe0e887e8ec4f3c, 6ba18eeef9a74b97ada822d2a00cfdf3; no voters:
I20260926 06:20:55.070290 12990 raft_consensus.cc:2781] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:20:55.070338 12990 raft_consensus.cc:686] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 1 LEADER]: Becoming Leader. State: Replica: 388fa46aa14f4d7fbfe0e887e8ec4f3c, State: Running, Role: LEADER
I20260926 06:20:55.070389 12990 consensus_queue.cc:237] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } }
I20260926 06:20:55.071017 12470 catalog_manager.cc:5495] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 388fa46aa14f4d7fbfe0e887e8ec4f3c reported cstate change: term changed from 0 to 1, leader changed from <none> to 388fa46aa14f4d7fbfe0e887e8ec4f3c (127.4.202.131). New cstate: current_term: 1 leader_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "388fa46aa14f4d7fbfe0e887e8ec4f3c" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 35099 } health_report { overall_health: HEALTHY } } }
I20260926 06:20:55.080299 12768 raft_consensus.cc:1261] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 1 FOLLOWER]: Refusing update from remote peer 388fa46aa14f4d7fbfe0e887e8ec4f3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:20:55.080366 12628 raft_consensus.cc:1261] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P ef281d65c9354fc483333fba732d64b4 [term 1 FOLLOWER]: Refusing update from remote peer 388fa46aa14f4d7fbfe0e887e8ec4f3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:20:55.080504 12990 consensus_queue.cc:1035] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ba18eeef9a74b97ada822d2a00cfdf3" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:20:55.080578 13011 consensus_queue.cc:1035] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef281d65c9354fc483333fba732d64b4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:20:55.089923 12470 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:20:55.089990 12470 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:20:55.090565 12470 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b4c397f387024f17b7d62d585e95e5cf: Sending DeleteTablet for 3 replicas of tablet 17b8f5254dec4c9ebb18ec77dce6bfd8
I20260926 06:20:55.090826 12608 tablet_service.cc:1514] Processing DeleteTablet for tablet 17b8f5254dec4c9ebb18ec77dce6bfd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:20:55 UTC) from {username='oryx', principal='oryx/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:47408
I20260926 06:20:55.090852 12888 tablet_service.cc:1514] Processing DeleteTablet for tablet 17b8f5254dec4c9ebb18ec77dce6bfd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:20:55 UTC) from {username='oryx', principal='oryx/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:57826
I20260926 06:20:55.090914 13052 tablet_replica.cc:330] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P ef281d65c9354fc483333fba732d64b4: stopping tablet replica
I20260926 06:20:55.090929 13051 tablet_replica.cc:330] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: stopping tablet replica
I20260926 06:20:55.090955 12748 tablet_service.cc:1514] Processing DeleteTablet for tablet 17b8f5254dec4c9ebb18ec77dce6bfd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:20:55 UTC) from {username='oryx', principal='oryx/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:58112
I20260926 06:20:55.090974 13051 raft_consensus.cc:2229] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:20:55.090973 13052 raft_consensus.cc:2229] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P ef281d65c9354fc483333fba732d64b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:20:55.091008 13052 raft_consensus.cc:2258] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P ef281d65c9354fc483333fba732d64b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:20:55.091022 13051 raft_consensus.cc:2258] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 388fa46aa14f4d7fbfe0e887e8ec4f3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:20:55.091027 13053 tablet_replica.cc:330] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 6ba18eeef9a74b97ada822d2a00cfdf3: stopping tablet replica
I20260926 06:20:55.091070 13053 raft_consensus.cc:2229] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:20:55.091101 13053 raft_consensus.cc:2258] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 6ba18eeef9a74b97ada822d2a00cfdf3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:20:55.091322 13051 ts_tablet_manager.cc:1905] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:20:55.091360 13053 ts_tablet_manager.cc:1905] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 6ba18eeef9a74b97ada822d2a00cfdf3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:20:55.091506 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 12543
I20260926 06:20:55.092998 13053 ts_tablet_manager.cc:1918] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 6ba18eeef9a74b97ada822d2a00cfdf3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 06:20:55.093048 13053 log.cc:1196] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 6ba18eeef9a74b97ada822d2a00cfdf3: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-1/wal/wals/17b8f5254dec4c9ebb18ec77dce6bfd8
I20260926 06:20:55.093066 13051 ts_tablet_manager.cc:1918] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 06:20:55.093118 13051 log.cc:1196] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790403591640726-4906-0/minicluster-data/ts-2/wal/wals/17b8f5254dec4c9ebb18ec77dce6bfd8
I20260926 06:20:55.093255 13053 ts_tablet_manager.cc:1939] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 6ba18eeef9a74b97ada822d2a00cfdf3: Deleting consensus metadata
I20260926 06:20:55.093343 13051 ts_tablet_manager.cc:1939] T 17b8f5254dec4c9ebb18ec77dce6bfd8 P 388fa46aa14f4d7fbfe0e887e8ec4f3c: Deleting consensus metadata
I20260926 06:20:55.093626 12458 catalog_manager.cc:4841] TS 6ba18eeef9a74b97ada822d2a00cfdf3 (127.4.202.130:44939): tablet 17b8f5254dec4c9ebb18ec77dce6bfd8 (table test-table [id=9626189142834af78b6741f431b8bffb]) successfully deleted
I20260926 06:20:55.094461 12458 catalog_manager.cc:4841] TS 388fa46aa14f4d7fbfe0e887e8ec4f3c (127.4.202.131:35099): tablet 17b8f5254dec4c9ebb18ec77dce6bfd8 (table test-table [id=9626189142834af78b6741f431b8bffb]) successfully deleted
W20260926 06:20:55.098752 12456 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.202.129:45759 (error 108)
I20260926 06:20:55.099260 13080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:55.099076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:45759 (local address 127.0.0.1:47602)
0926 06:20:55.099173 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:55.099206 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.129:45759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":41,"threads_started":1}
I20260926 06:20:55.099313 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 12683
W20260926 06:20:55.102694 12456 catalog_manager.cc:4584] TS ef281d65c9354fc483333fba732d64b4 (127.4.202.129:45759): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 17b8f5254dec4c9ebb18ec77dce6bfd8: Network error: Client connection negotiation failed: client connection to 127.4.202.129:45759: connect: Connection refused (error 111)
I20260926 06:20:55.104574 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 12823
I20260926 06:20:55.110059 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 12439
2026-09-26T06:20:55Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2834 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:20:55 dist-test-slave-pd0t krb5kdc[13083](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:20:55 dist-test-slave-pd0t krb5kdc[13083](info): set up 2 sockets
Sep 26 06:20:55 dist-test-slave-pd0t krb5kdc[13083](info): commencing operation
krb5kdc: starting...
W20260926 06:20:56.139257 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:20:56 dist-test-slave-pd0t krb5kdc[13083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:20:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:20:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.202.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
I20260926 06:20:56.249138 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:33355
--webserver_interface=127.4.202.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:41207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:33355
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:56.327225 13096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:56.327427 13096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:56.327463 13096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:56.328583 13096 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:20:56.328644 13096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:56.328670 13096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:56.328701 13096 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:20:56.328723 13096 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:20:56.330229 13096 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:41207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:33355
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:33355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:56.330490 13096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:56.330725 13096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:56.335237 13101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:56.335251 13102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:56.335240 13104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:56.335340 13096 server_base.cc:1034] running on GCE node
I20260926 06:20:56.335664 13096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:56.335932 13096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:56.337069 13096 hybrid_clock.cc:648] HybridClock initialized: now 1790403656337055 us; error 43 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-26T06:20:56Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1234 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:20:56 dist-test-slave-pd0t krb5kdc[13109](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:20:56 dist-test-slave-pd0t krb5kdc[13109](info): set up 2 sockets
Sep 26 06:20:56 dist-test-slave-pd0t krb5kdc[13109](info): commencing operation
krb5kdc: starting...
W20260926 06:20:57.378923 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:20:57 dist-test-slave-pd0t krb5kdc[13109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:20:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:20:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.202.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
I20260926 06:20:57.493796 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:37671
--webserver_interface=127.4.202.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:37847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:37671
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:57.572752 13122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:57.572976 13122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:57.573012 13122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:57.574097 13122 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:20:57.574155 13122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:57.574178 13122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:57.574199 13122 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:20:57.574224 13122 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:20:57.575609 13122 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:37847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:37671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:37671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:57.575868 13122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:57.576067 13122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:57.580610 13130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:57.580652 13128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:57.580652 13127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:57.580718 13122 server_base.cc:1034] running on GCE node
I20260926 06:20:57.580996 13122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:57.581281 13122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:57.582422 13122 hybrid_clock.cc:648] HybridClock initialized: now 1790403657582390 us; error 53 us; skew 500 ppm
Sep 26 06:20:57 dist-test-slave-pd0t krb5kdc[13109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403657, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:57.584815 13122 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:20:57.585213 13122 webserver.cc:460] Webserver started at http://127.4.202.190:37765/ using document root <none> and password file <none>
I20260926 06:20:57.585428 13122 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:57.585490 13122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:57.585626 13122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:57.586649 13122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "918e4ff7e7784a1c85a06c327c108fa2"
format_stamp: "Formatted at 2026-09-26 06:20:57 on dist-test-slave-pd0t"
server_key: "88c68f1b182911e9c4b79530145b70ed"
server_key_iv: "e16bc35413b9df163786d2728d203275"
server_key_version: "encryptionkey@0"
I20260926 06:20:57.587000 13122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "918e4ff7e7784a1c85a06c327c108fa2"
format_stamp: "Formatted at 2026-09-26 06:20:57 on dist-test-slave-pd0t"
server_key: "88c68f1b182911e9c4b79530145b70ed"
server_key_iv: "e16bc35413b9df163786d2728d203275"
server_key_version: "encryptionkey@0"
I20260926 06:20:57.588343 13122 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:20:57.589051 13137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:57.589226 13122 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:20:57.589315 13122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "918e4ff7e7784a1c85a06c327c108fa2"
format_stamp: "Formatted at 2026-09-26 06:20:57 on dist-test-slave-pd0t"
server_key: "88c68f1b182911e9c4b79530145b70ed"
server_key_iv: "e16bc35413b9df163786d2728d203275"
server_key_version: "encryptionkey@0"
I20260926 06:20:57.589414 13122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:57.598120 13122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:57.600220 13122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:57.600368 13122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:57.604501 13122 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:37671
I20260926 06:20:57.604533 13189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:37671 every 8 connection(s)
I20260926 06:20:57.604835 13122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:20:57.605659 13190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:20:57.607774 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 13122
I20260926 06:20:57.607815 13190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2: Bootstrap starting.
I20260926 06:20:57.607884 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:20:57.608052 4906 external_mini_cluster.cc:1428] Setting key a2eca53132033bc3ee9dbf1a3e715ac7
I20260926 06:20:57.608403 13190 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2: Neither blocks nor log segments found. Creating new log.
I20260926 06:20:57.608975 13190 log.cc:824] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2: Log is configured to *not* fsync() on all Append() calls
I20260926 06:20:57.609697 13190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2: No bootstrap required, opened a new log
I20260926 06:20:57.612071 13190 raft_consensus.cc:348] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "918e4ff7e7784a1c85a06c327c108fa2" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 37671 } }
I20260926 06:20:57.612203 13190 raft_consensus.cc:374] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:20:57.612224 13190 raft_consensus.cc:729] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 918e4ff7e7784a1c85a06c327c108fa2, State: Initialized, Role: FOLLOWER
I20260926 06:20:57.612393 13190 consensus_queue.cc:260] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "918e4ff7e7784a1c85a06c327c108fa2" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 37671 } }
I20260926 06:20:57.612461 13190 raft_consensus.cc:388] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:20:57.612483 13190 raft_consensus.cc:482] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:20:57.612572 13190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:20:57.613065 13190 raft_consensus.cc:504] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "918e4ff7e7784a1c85a06c327c108fa2" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 37671 } }
I20260926 06:20:57.613178 13190 leader_election.cc:302] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 918e4ff7e7784a1c85a06c327c108fa2; no voters:
I20260926 06:20:57.613363 13190 leader_election.cc:288] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:20:57.613387 13195 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:20:57.613492 13195 raft_consensus.cc:686] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [term 1 LEADER]: Becoming Leader. State: Replica: 918e4ff7e7784a1c85a06c327c108fa2, State: Running, Role: LEADER
I20260926 06:20:57.613569 13195 consensus_queue.cc:237] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "918e4ff7e7784a1c85a06c327c108fa2" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 37671 } }
I20260926 06:20:57.613608 13190 sys_catalog.cc:564] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:20:57.614530 13196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "918e4ff7e7784a1c85a06c327c108fa2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "918e4ff7e7784a1c85a06c327c108fa2" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 37671 } } }
I20260926 06:20:57.614557 13197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 918e4ff7e7784a1c85a06c327c108fa2. Latest consensus state: current_term: 1 leader_uuid: "918e4ff7e7784a1c85a06c327c108fa2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "918e4ff7e7784a1c85a06c327c108fa2" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 37671 } } }
I20260926 06:20:57.614651 13196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [sys.catalog]: This master's current role is: LEADER
I20260926 06:20:57.614658 13197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [sys.catalog]: This master's current role is: LEADER
Sep 26 06:20:57 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
W20260926 06:20:57.615379 13210 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:20:57.615604 13210 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:20:57.615983 13206 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:20:57.616190 13206 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:20:57.617791 13206 catalog_manager.cc:1347] Generated new cluster ID: dcf4f604431744eaa717a815afb55b24
I20260926 06:20:57.617873 13206 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:20:57.619477 13193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:57.608583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41830 (local address 127.4.202.190:37671)
0926 06:20:57.608737 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:57.608746 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:57.608791 (+ 45us) server_negotiation.cc:407] Connection header received
0926 06:20:57.609347 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:57.609354 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:57.609432 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:57.609571 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:57.610191 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:57.611608 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:57.612562 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:57.613034 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:57.613151 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:57.615905 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:57.615909 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:57.615910 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:57.615923 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:57.617871 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:57.618254 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:57.618263 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:57.618263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:57.618354 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:57.618615 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:57.618616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:57.618616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:57.618889 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:57.619034 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:57.619155 (+ 121us) server_negotiation.cc:299] Negotiation successful
0926 06:20:57.619276 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":72,"threads_started":1}
I20260926 06:20:57.627828 13206 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:20:57.628546 13206 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:20:57.639441 13206 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2: Generated new TSK 0
I20260926 06:20:57.639623 13206 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.202.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
I20260926 06:20:57.663803 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.129:0
--local_ip_for_outbound_sockets=127.4.202.129
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:37671
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:37847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:57.743132 13216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:57.743314 13216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:57.743338 13216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:57.744432 13216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:57.744474 13216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:57.744520 13216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.129
I20260926 06:20:57.745967 13216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:37847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:37671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.129
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:57.746179 13216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:57.746402 13216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:57.751219 13222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:57.751235 13221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:57.751328 13224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:57.751394 13216 server_base.cc:1034] running on GCE node
I20260926 06:20:57.751663 13216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:57.751943 13216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:57.753072 13216 hybrid_clock.cc:648] HybridClock initialized: now 1790403657753057 us; error 36 us; skew 500 ppm
Sep 26 06:20:57 dist-test-slave-pd0t krb5kdc[13109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403657, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:57.755551 13216 init.cc:377] Logged in from keytab as kudu/127.4.202.129@KRBTEST.COM (short username kudu)
I20260926 06:20:57.755947 13216 webserver.cc:460] Webserver started at http://127.4.202.129:41679/ using document root <none> and password file <none>
I20260926 06:20:57.756145 13216 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:57.756187 13216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:57.756326 13216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:57.757258 13216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-0/data/instance:
uuid: "67396869dab64069abcf718c69b1dc73"
format_stamp: "Formatted at 2026-09-26 06:20:57 on dist-test-slave-pd0t"
server_key: "3a58d7ca7b5eed736259a5c8bc83c7c6"
server_key_iv: "380a6d02981b46266ac0bb21077b83ba"
server_key_version: "encryptionkey@0"
I20260926 06:20:57.757632 13216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance:
uuid: "67396869dab64069abcf718c69b1dc73"
format_stamp: "Formatted at 2026-09-26 06:20:57 on dist-test-slave-pd0t"
server_key: "3a58d7ca7b5eed736259a5c8bc83c7c6"
server_key_iv: "380a6d02981b46266ac0bb21077b83ba"
server_key_version: "encryptionkey@0"
I20260926 06:20:57.758879 13216 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:20:57.759547 13231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:57.759698 13216 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:20:57.759811 13216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-0/wal
uuid: "67396869dab64069abcf718c69b1dc73"
format_stamp: "Formatted at 2026-09-26 06:20:57 on dist-test-slave-pd0t"
server_key: "3a58d7ca7b5eed736259a5c8bc83c7c6"
server_key_iv: "380a6d02981b46266ac0bb21077b83ba"
server_key_version: "encryptionkey@0"
I20260926 06:20:57.759877 13216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:57.773321 13216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:57.775450 13216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:57.775594 13216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:57.775784 13216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:20:57.776070 13216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:20:57.776139 13216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:57.776175 13216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:20:57.776204 13216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:57.781656 13216 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.129:36277
I20260926 06:20:57.781705 13344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.129:36277 every 8 connection(s)
I20260926 06:20:57.781978 13216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
Sep 26 06:20:57 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403657, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:20:57.787761 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 13216
I20260926 06:20:57.787878 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance
I20260926 06:20:57.788072 4906 external_mini_cluster.cc:1428] Setting key 1072fde05174c75948738fe296a9edec
I20260926 06:20:57.790916 13193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:57.782558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:42745 (local address 127.4.202.190:37671)
0926 06:20:57.782643 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:57.782646 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:57.782814 (+ 168us) server_negotiation.cc:407] Connection header received
0926 06:20:57.783237 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:57.783238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:57.783248 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:57.783290 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:57.784119 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:57.784621 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:57.785599 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:57.785795 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:57.785851 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:57.787857 (+ 2006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:57.787859 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:57.787859 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:57.787869 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:57.789518 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:57.789866 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:57.789867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:57.789867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:57.789928 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:57.790237 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:57.790238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:57.790238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:57.790362 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:57.790440 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:57.790720 (+ 280us) server_negotiation.cc:299] Negotiation successful
0926 06:20:57.790804 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 06:20:57.791115 13347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:57.782531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.129:42745)
0926 06:20:57.782706 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:57.782738 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:57.783154 (+ 416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:57.783379 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:57.783383 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:57.783886 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:57.784050 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:57.784054 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:57.784695 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:57.784696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:57.785534 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:57.785536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:57.785862 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:57.785863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:57.785949 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:57.786354 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:57.786361 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:57.787736 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:57.789602 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:57.789607 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:57.789609 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:57.789803 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:57.790044 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:57.790045 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:57.790045 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:57.790153 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:57.790429 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:57.790435 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:57.790564 (+ 129us) client_negotiation.cc:769] Sending connection context
0926 06:20:57.790699 (+ 135us) client_negotiation.cc:240] Negotiation successful
0926 06:20:57.790841 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":110,"threads_started":1}
I20260926 06:20:57.791543 13345 heartbeater.cc:347] Connected to a master server at 127.4.202.190:37671
I20260926 06:20:57.791626 13345 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:57.791821 13345 heartbeater.cc:511] Master 127.4.202.190:37671 requested a full tablet report, sending...
I20260926 06:20:57.792294 13154 ts_manager.cc:194] Registered new tserver with Master: 67396869dab64069abcf718c69b1dc73 (127.4.202.129:36277)
I20260926 06:20:57.792941 13154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.129@KRBTEST.COM'} at 127.4.202.129:42745
WARNING: no policy specified for kudu/127.4.202.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
I20260926 06:20:57.813062 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.130:0
--local_ip_for_outbound_sockets=127.4.202.130
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:37671
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:37847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:57.890262 13350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:57.890450 13350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:57.890472 13350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:57.891549 13350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:57.891597 13350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:57.891620 13350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.130
I20260926 06:20:57.892997 13350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:37847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:37671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.130
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:57.893208 13350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:57.893386 13350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:57.897997 13355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:57.898002 13358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:57.898065 13356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:57.898202 13350 server_base.cc:1034] running on GCE node
I20260926 06:20:57.898386 13350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:57.898681 13350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:57.899832 13350 hybrid_clock.cc:648] HybridClock initialized: now 1790403657899816 us; error 47 us; skew 500 ppm
Sep 26 06:20:57 dist-test-slave-pd0t krb5kdc[13109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403657, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:57.902007 13350 init.cc:377] Logged in from keytab as kudu/127.4.202.130@KRBTEST.COM (short username kudu)
I20260926 06:20:57.902405 13350 webserver.cc:460] Webserver started at http://127.4.202.130:40695/ using document root <none> and password file <none>
I20260926 06:20:57.902664 13350 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:57.902714 13350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:57.902832 13350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:57.903760 13350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-1/data/instance:
uuid: "589fced5765941eb8584cf8a622f731a"
format_stamp: "Formatted at 2026-09-26 06:20:57 on dist-test-slave-pd0t"
server_key: "5139e208aa612e24cf0693a1a9b8722d"
server_key_iv: "f4de6c29f33a50e2eb621ec681108be7"
server_key_version: "encryptionkey@0"
I20260926 06:20:57.904089 13350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance:
uuid: "589fced5765941eb8584cf8a622f731a"
format_stamp: "Formatted at 2026-09-26 06:20:57 on dist-test-slave-pd0t"
server_key: "5139e208aa612e24cf0693a1a9b8722d"
server_key_iv: "f4de6c29f33a50e2eb621ec681108be7"
server_key_version: "encryptionkey@0"
I20260926 06:20:57.905426 13350 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:20:57.906116 13365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:57.906294 13350 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:20:57.906401 13350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-1/wal
uuid: "589fced5765941eb8584cf8a622f731a"
format_stamp: "Formatted at 2026-09-26 06:20:57 on dist-test-slave-pd0t"
server_key: "5139e208aa612e24cf0693a1a9b8722d"
server_key_iv: "f4de6c29f33a50e2eb621ec681108be7"
server_key_version: "encryptionkey@0"
I20260926 06:20:57.906452 13350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:57.931846 13350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:57.933665 13350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:57.933765 13350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:57.933935 13350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:20:57.934243 13350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:20:57.934274 13350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:57.934295 13350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:20:57.934347 13350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:57.940260 13350 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.130:33563
I20260926 06:20:57.940301 13478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.130:33563 every 8 connection(s)
I20260926 06:20:57.940601 13350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
Sep 26 06:20:57 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403657, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:20:57.947414 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 13350
I20260926 06:20:57.947513 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance
I20260926 06:20:57.947691 4906 external_mini_cluster.cc:1428] Setting key 7b13c822804b040ee52cb98b83925807
I20260926 06:20:57.949483 13481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:57.941199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.130:56765)
0926 06:20:57.941380 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:57.941421 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:57.941854 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:57.942105 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:57.942108 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:57.942709 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:57.942915 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:57.942920 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:57.943565 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:57.943566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:57.944345 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:57.944346 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:57.944653 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:57.944654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:57.944745 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:57.945146 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:57.945154 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:57.946433 (+ 1279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:57.948149 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:57.948155 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:57.948157 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:57.948450 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:57.948664 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:57.948664 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:57.948665 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:57.948757 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:57.949037 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:57.949041 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:57.949146 (+ 105us) client_negotiation.cc:769] Sending connection context
0926 06:20:57.949225 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 06:20:57.949337 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":95,"threads_started":1}
I20260926 06:20:57.949504 13193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:57.941236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:56765 (local address 127.4.202.190:37671)
0926 06:20:57.941292 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:57.941294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:57.941500 (+ 206us) server_negotiation.cc:407] Connection header received
0926 06:20:57.941961 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:57.941963 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:57.941973 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:57.942033 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:57.942998 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:57.943492 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:57.944417 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:57.944587 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:57.944637 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:57.946514 (+ 1877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:57.946515 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:57.946515 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:57.946522 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:57.948078 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:57.948525 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:57.948527 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:57.948527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:57.948589 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:57.948816 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:57.948817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:57.948817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:57.948974 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:57.949044 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:57.949261 (+ 217us) server_negotiation.cc:299] Negotiation successful
0926 06:20:57.949343 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 06:20:57.949906 13479 heartbeater.cc:347] Connected to a master server at 127.4.202.190:37671
I20260926 06:20:57.949992 13479 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:57.950169 13479 heartbeater.cc:511] Master 127.4.202.190:37671 requested a full tablet report, sending...
I20260926 06:20:57.950525 13154 ts_manager.cc:194] Registered new tserver with Master: 589fced5765941eb8584cf8a622f731a (127.4.202.130:33563)
I20260926 06:20:57.951050 13154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.130@KRBTEST.COM'} at 127.4.202.130:56765
WARNING: no policy specified for kudu/127.4.202.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
I20260926 06:20:57.973006 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.131:0
--local_ip_for_outbound_sockets=127.4.202.131
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:37671
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:37847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:58.051868 13484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:58.052095 13484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:58.052132 13484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:58.053272 13484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:58.053344 13484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:58.053391 13484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.131
I20260926 06:20:58.054855 13484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:37847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:37671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.131
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:58.055100 13484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:58.055297 13484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:58.060112 13489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:58.060161 13484 server_base.cc:1034] running on GCE node
W20260926 06:20:58.060129 13490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:58.060109 13492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:58.060458 13484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:58.060642 13484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:58.061770 13484 hybrid_clock.cc:648] HybridClock initialized: now 1790403658061756 us; error 30 us; skew 500 ppm
Sep 26 06:20:58 dist-test-slave-pd0t krb5kdc[13109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403658, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:58.064102 13484 init.cc:377] Logged in from keytab as kudu/127.4.202.131@KRBTEST.COM (short username kudu)
I20260926 06:20:58.064476 13484 webserver.cc:460] Webserver started at http://127.4.202.131:38041/ using document root <none> and password file <none>
I20260926 06:20:58.064646 13484 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:58.064684 13484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:58.064767 13484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:20:58.065662 13484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-2/data/instance:
uuid: "f8cbf0e48cdb43108d27809a6b0a64aa"
format_stamp: "Formatted at 2026-09-26 06:20:58 on dist-test-slave-pd0t"
server_key: "413870443d8d4def633c92ec8142bd25"
server_key_iv: "4671796d970bf3a35fed2f6acfe0eda7"
server_key_version: "encryptionkey@0"
I20260926 06:20:58.065932 13484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance:
uuid: "f8cbf0e48cdb43108d27809a6b0a64aa"
format_stamp: "Formatted at 2026-09-26 06:20:58 on dist-test-slave-pd0t"
server_key: "413870443d8d4def633c92ec8142bd25"
server_key_iv: "4671796d970bf3a35fed2f6acfe0eda7"
server_key_version: "encryptionkey@0"
I20260926 06:20:58.067174 13484 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:20:58.067814 13499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:58.067962 13484 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:20:58.068037 13484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-2/wal
uuid: "f8cbf0e48cdb43108d27809a6b0a64aa"
format_stamp: "Formatted at 2026-09-26 06:20:58 on dist-test-slave-pd0t"
server_key: "413870443d8d4def633c92ec8142bd25"
server_key_iv: "4671796d970bf3a35fed2f6acfe0eda7"
server_key_version: "encryptionkey@0"
I20260926 06:20:58.068086 13484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:58.081818 13484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:58.083592 13484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:58.083698 13484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:58.083851 13484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:20:58.084070 13484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:20:58.084098 13484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:58.084120 13484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:20:58.084131 13484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:58.089607 13484 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.131:36097
I20260926 06:20:58.089679 13612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.131:36097 every 8 connection(s)
I20260926 06:20:58.089956 13484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
Sep 26 06:20:58 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403658, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:20:58.096998 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 13484
I20260926 06:20:58.097126 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance
I20260926 06:20:58.097316 4906 external_mini_cluster.cc:1428] Setting key 6b125a6e17a767c54916b8c6ab68970f
I20260926 06:20:58.099123 13193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:58.090717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:38375 (local address 127.4.202.190:37671)
0926 06:20:58.090774 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:58.090776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:58.090990 (+ 214us) server_negotiation.cc:407] Connection header received
0926 06:20:58.091504 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:58.091505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:58.091517 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:58.091573 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:58.092490 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.093009 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.094041 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.094231 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.094330 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:58.096281 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:58.096283 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:58.096283 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:58.096289 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:58.097841 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:58.098314 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:58.098315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:58.098316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:58.098376 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:58.098553 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:58.098554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:58.098554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:58.098708 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:58.098759 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:58.098986 (+ 227us) server_negotiation.cc:299] Negotiation successful
0926 06:20:58.099062 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 06:20:58.099232 13615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:58.090710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.131:38375)
0926 06:20:58.090883 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:58.090916 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:58.091386 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:58.091643 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:58.091647 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:58.092193 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:58.092383 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:58.092387 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.093097 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.093099 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:58.093945 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:58.093947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.094329 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.094330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:58.094452 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:58.094910 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:58.094918 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:58.096198 (+ 1280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:20:58.097942 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:58.097948 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:58.097950 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:58.098240 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:58.098415 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:20:58.098415 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:20:58.098416 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:20:58.098506 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:20:58.098751 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:20:58.098754 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:20:58.098856 (+ 102us) client_negotiation.cc:769] Sending connection context
0926 06:20:58.098941 (+ 85us) client_negotiation.cc:240] Negotiation successful
0926 06:20:58.099050 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":80,"threads_started":1}
I20260926 06:20:58.099649 13613 heartbeater.cc:347] Connected to a master server at 127.4.202.190:37671
I20260926 06:20:58.099725 13613 heartbeater.cc:464] Registering TS with master...
I20260926 06:20:58.099858 13613 heartbeater.cc:511] Master 127.4.202.190:37671 requested a full tablet report, sending...
I20260926 06:20:58.100189 13154 ts_manager.cc:194] Registered new tserver with Master: f8cbf0e48cdb43108d27809a6b0a64aa (127.4.202.131:36097)
I20260926 06:20:58.100512 13154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.131@KRBTEST.COM'} at 127.4.202.131:38375
I20260926 06:20:58.109009 4906 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.4.202.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.202.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.202.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.129.keytab.
Entry for principal oryx/127.4.202.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.129.keytab.
WARNING: no policy specified for oryx/127.4.202.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.202.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.202.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.130.keytab.
Entry for principal oryx/127.4.202.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.130.keytab.
WARNING: no policy specified for oryx/127.4.202.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.202.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.202.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.131.keytab.
Entry for principal oryx/127.4.202.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.131.keytab.
I20260926 06:20:58.173871 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 13216
I20260926 06:20:58.178166 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 13350
I20260926 06:20:58.182268 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 13484
I20260926 06:20:58.186357 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 13122
I20260926 06:20:58.191093 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:37671
--webserver_interface=127.4.202.190
--webserver_port=37765
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:37847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:37671
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:20:58.268617 13622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:58.268863 13622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:58.268885 13622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:58.269991 13622 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:20:58.270028 13622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:58.270042 13622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:58.270054 13622 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:20:58.270066 13622 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:20:58.271466 13622 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:37847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:37671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:37671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=37765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:58.271653 13622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:58.271808 13622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:58.276448 13628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:58.276525 13627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:58.276672 13622 server_base.cc:1034] running on GCE node
W20260926 06:20:58.276525 13630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:58.277012 13622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:58.277276 13622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:58.278435 13622 hybrid_clock.cc:648] HybridClock initialized: now 1790403658278422 us; error 23 us; skew 500 ppm
Sep 26 06:20:58 dist-test-slave-pd0t krb5kdc[13109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403658, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:58.280750 13622 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:20:58.281142 13622 webserver.cc:460] Webserver started at http://127.4.202.190:37765/ using document root <none> and password file <none>
I20260926 06:20:58.281363 13622 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:58.281441 13622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:58.282747 13622 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:20:58.283346 13637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:58.283501 13622 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:58.283591 13622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "918e4ff7e7784a1c85a06c327c108fa2"
format_stamp: "Formatted at 2026-09-26 06:20:57 on dist-test-slave-pd0t"
server_key: "88c68f1b182911e9c4b79530145b70ed"
server_key_iv: "e16bc35413b9df163786d2728d203275"
server_key_version: "encryptionkey@0"
I20260926 06:20:58.283903 13622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:58.304154 13622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:58.306300 13622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:58.306455 13622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:58.310141 13622 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:37671
I20260926 06:20:58.310205 13689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:37671 every 8 connection(s)
I20260926 06:20:58.310566 13622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:20:58.311964 13690 sys_catalog.cc:263] Verifying existing consensus state
I20260926 06:20:58.312606 13690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2: Bootstrap starting.
I20260926 06:20:58.315284 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 13622
I20260926 06:20:58.315606 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.129:36277
--local_ip_for_outbound_sockets=127.4.202.129
--tserver_master_addrs=127.4.202.190:37671
--webserver_port=41679
--webserver_interface=127.4.202.129
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:37847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.4.202.129
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:20:58.321033 13690 log.cc:824] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2: Log is configured to *not* fsync() on all Append() calls
I20260926 06:20:58.322537 13690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 06:20:58.322835 13690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2: Bootstrap complete.
I20260926 06:20:58.324939 13690 raft_consensus.cc:348] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "918e4ff7e7784a1c85a06c327c108fa2" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 37671 } }
I20260926 06:20:58.325142 13690 raft_consensus.cc:729] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 918e4ff7e7784a1c85a06c327c108fa2, State: Initialized, Role: FOLLOWER
I20260926 06:20:58.325292 13690 consensus_queue.cc:260] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "918e4ff7e7784a1c85a06c327c108fa2" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 37671 } }
I20260926 06:20:58.325357 13690 raft_consensus.cc:388] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:20:58.325408 13690 raft_consensus.cc:482] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:20:58.325459 13690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [term 1 FOLLOWER]: Advancing to term 2
I20260926 06:20:58.326399 13690 raft_consensus.cc:504] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "918e4ff7e7784a1c85a06c327c108fa2" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 37671 } }
I20260926 06:20:58.326529 13690 leader_election.cc:302] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 918e4ff7e7784a1c85a06c327c108fa2; no voters:
I20260926 06:20:58.326721 13690 leader_election.cc:288] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 06:20:58.326758 13694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 06:20:58.326907 13694 raft_consensus.cc:686] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [term 2 LEADER]: Becoming Leader. State: Replica: 918e4ff7e7784a1c85a06c327c108fa2, State: Running, Role: LEADER
I20260926 06:20:58.327087 13694 consensus_queue.cc:237] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "918e4ff7e7784a1c85a06c327c108fa2" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 37671 } }
I20260926 06:20:58.326994 13690 sys_catalog.cc:564] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:20:58.327826 13695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 918e4ff7e7784a1c85a06c327c108fa2. Latest consensus state: current_term: 2 leader_uuid: "918e4ff7e7784a1c85a06c327c108fa2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "918e4ff7e7784a1c85a06c327c108fa2" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 37671 } } }
I20260926 06:20:58.327907 13695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [sys.catalog]: This master's current role is: LEADER
I20260926 06:20:58.328075 13696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "918e4ff7e7784a1c85a06c327c108fa2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "918e4ff7e7784a1c85a06c327c108fa2" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 37671 } } }
I20260926 06:20:58.328159 13696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2 [sys.catalog]: This master's current role is: LEADER
I20260926 06:20:58.328075 13704 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:20:58.328680 13704 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:20:58.328801 13704 catalog_manager.cc:1259] Loaded cluster ID: dcf4f604431744eaa717a815afb55b24
I20260926 06:20:58.328903 13704 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:20:58.330130 13704 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:20:58.330313 13704 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 918e4ff7e7784a1c85a06c327c108fa2: Loaded TSK: 0
I20260926 06:20:58.330531 13704 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 06:20:58.394773 13692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:58.394999 13692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:58.395038 13692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:58.396173 13692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:58.396232 13692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:58.396276 13692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.129
I20260926 06:20:58.397686 13692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:37847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.129.keytab
--principal=oryx/127.4.202.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.129:36277
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.202.129
--webserver_port=41679
--tserver_master_addrs=127.4.202.190:37671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.129
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:58.397941 13692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:58.398159 13692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:58.403932 13719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:58.403939 13716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:58.403944 13717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:58.404166 13692 server_base.cc:1034] running on GCE node
I20260926 06:20:58.404366 13692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:58.404600 13692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:58.405772 13692 hybrid_clock.cc:648] HybridClock initialized: now 1790403658405760 us; error 32 us; skew 500 ppm
Sep 26 06:20:58 dist-test-slave-pd0t krb5kdc[13109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403658, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:58.408384 13692 init.cc:377] Logged in from keytab as oryx/127.4.202.129@KRBTEST.COM (short username oryx)
I20260926 06:20:58.408896 13692 webserver.cc:460] Webserver started at http://127.4.202.129:41679/ using document root <none> and password file <none>
I20260926 06:20:58.409133 13692 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:58.409214 13692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:58.410884 13692 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 06:20:58.411530 13726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:58.411736 13692 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:20:58.411823 13692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-0/wal
uuid: "67396869dab64069abcf718c69b1dc73"
format_stamp: "Formatted at 2026-09-26 06:20:57 on dist-test-slave-pd0t"
server_key: "3a58d7ca7b5eed736259a5c8bc83c7c6"
server_key_iv: "380a6d02981b46266ac0bb21077b83ba"
server_key_version: "encryptionkey@0"
I20260926 06:20:58.412210 13692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:58.432493 13692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:58.434744 13692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:58.434887 13692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:58.435111 13692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:20:58.435438 13692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:20:58.435485 13692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:58.435545 13692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:20:58.435578 13692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:58.442629 13692 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.129:36277
I20260926 06:20:58.442673 13839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.129:36277 every 8 connection(s)
I20260926 06:20:58.443080 13692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
Sep 26 06:20:58 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:20:58.450508 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 13692
I20260926 06:20:58.450873 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.130:33563
--local_ip_for_outbound_sockets=127.4.202.130
--tserver_master_addrs=127.4.202.190:37671
--webserver_port=40695
--webserver_interface=127.4.202.130
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:37847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.4.202.130
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 06:20:58 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:20:58.452179 13842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:58.443705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.129:33871)
0926 06:20:58.443896 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:58.443939 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:58.444531 (+ 592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:58.444925 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:58.444929 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:58.445589 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:58.445820 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:58.445825 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.446983 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.446985 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:58.448166 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:58.448169 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.448742 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.448743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:58.448870 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:58.449323 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:58.449332 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:58.451964 (+ 2632us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":119,"threads_started":1}
W20260926 06:20:58.452311 13843 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:20:58.443812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:33871 (local address 127.4.202.190:37671)
0926 06:20:58.443976 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:58.443984 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:58.444018 (+ 34us) server_negotiation.cc:407] Connection header received
0926 06:20:58.444715 (+ 697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:58.444720 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:58.444759 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:58.444855 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:58.445924 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.446899 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.448252 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.448661 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.448737 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:58.452137 (+ 3400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.129:33871: BlockingRecv error: recv got EOF from 127.4.202.129:33871 (error 108)
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":69,"threads_started":1}
W20260926 06:20:58.452777 13840 heartbeater.cc:644] Failed to heartbeat to 127.4.202.190:37671 (0 consecutive failures): Not authorized: Failed to ping master at 127.4.202.190:37671: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Sep 26 06:20:58 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:20:58 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:20:58.458168 13842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:58.452995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.129:37945)
0926 06:20:58.453087 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:58.453097 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:58.453139 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:58.453268 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:58.453269 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:58.453420 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:58.453465 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:58.453466 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.454397 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.454398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:58.455226 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:58.455229 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.455770 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.455771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:58.455835 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:58.456273 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:58.456278 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:58.458102 (+ 1824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":78}
W20260926 06:20:58.458465 13843 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:20:58.453049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:37945 (local address 127.4.202.190:37671)
0926 06:20:58.453085 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:58.453087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:58.453128 (+ 41us) server_negotiation.cc:407] Connection header received
0926 06:20:58.453186 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:58.453187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:58.453198 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:58.453257 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:58.453535 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.454305 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.455339 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.455662 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.455720 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:58.458400 (+ 2680us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.129:37945: BlockingRecv error: recv got EOF from 127.4.202.129:37945 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Sep 26 06:20:58 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:20:58 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:20:58.463644 13842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:58.458878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.129:57283)
0926 06:20:58.458949 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:58.458958 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:58.458989 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:58.459149 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:58.459149 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:58.459277 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:58.459305 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:58.459306 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.460225 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.460225 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:58.460974 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:58.460975 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.461411 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.461412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:58.461476 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:58.461880 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:58.461887 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:58.463596 (+ 1709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":59}
W20260926 06:20:58.463734 13843 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:20:58.458975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:57283 (local address 127.4.202.190:37671)
0926 06:20:58.459018 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:58.459021 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:58.459030 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:20:58.459052 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:58.459053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:58.459063 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:58.459145 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:58.459379 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.460149 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.461048 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.461336 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.461401 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:58.463679 (+ 2278us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.129:57283: BlockingRecv error: recv got EOF from 127.4.202.129:57283 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260926 06:20:58.463892 13840 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 06:20:58.532985 13844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:58.533186 13844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:58.533224 13844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:58.534392 13844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:58.534452 13844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:58.534497 13844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.130
I20260926 06:20:58.535992 13844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:37847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.130.keytab
--principal=oryx/127.4.202.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.130:33563
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.202.130
--webserver_port=40695
--tserver_master_addrs=127.4.202.190:37671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.130
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:58.536239 13844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:58.536453 13844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:58.541316 13849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:58.541440 13850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:58.541301 13852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:58.541301 13844 server_base.cc:1034] running on GCE node
I20260926 06:20:58.541776 13844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:58.542001 13844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:58.543152 13844 hybrid_clock.cc:648] HybridClock initialized: now 1790403658543137 us; error 26 us; skew 500 ppm
Sep 26 06:20:58 dist-test-slave-pd0t krb5kdc[13109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403658, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:58.545599 13844 init.cc:377] Logged in from keytab as oryx/127.4.202.130@KRBTEST.COM (short username oryx)
I20260926 06:20:58.546034 13844 webserver.cc:460] Webserver started at http://127.4.202.130:40695/ using document root <none> and password file <none>
I20260926 06:20:58.546254 13844 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:58.546337 13844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:58.547636 13844 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:20:58.548151 13859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:58.548303 13844 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:20:58.548413 13844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-1/wal
uuid: "589fced5765941eb8584cf8a622f731a"
format_stamp: "Formatted at 2026-09-26 06:20:57 on dist-test-slave-pd0t"
server_key: "5139e208aa612e24cf0693a1a9b8722d"
server_key_iv: "f4de6c29f33a50e2eb621ec681108be7"
server_key_version: "encryptionkey@0"
I20260926 06:20:58.548717 13844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:58.562247 13844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:58.564599 13844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:58.564769 13844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:58.564990 13844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:20:58.565294 13844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:20:58.565347 13844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:58.565392 13844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:20:58.565423 13844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:58.571908 13844 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.130:33563
I20260926 06:20:58.571954 13972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.130:33563 every 8 connection(s)
I20260926 06:20:58.572304 13844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
I20260926 06:20:58.576012 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 13844
I20260926 06:20:58.576282 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.131:36097
--local_ip_for_outbound_sockets=127.4.202.131
--tserver_master_addrs=127.4.202.190:37671
--webserver_port=38041
--webserver_interface=127.4.202.131
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:37847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.4.202.131
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 06:20:58 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:20:58 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
W20260926 06:20:58.579746 13843 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:20:58.572902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:43903 (local address 127.4.202.190:37671)
0926 06:20:58.572963 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:58.572965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:58.573145 (+ 180us) server_negotiation.cc:407] Connection header received
0926 06:20:58.573594 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:58.573595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:58.573608 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:58.573658 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:58.574558 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.575074 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.576105 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.576367 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.576436 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:58.579702 (+ 3266us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.130:43903: BlockingRecv error: recv got EOF from 127.4.202.130:43903 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 06:20:58.579865 13975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:58.572892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.130:43903)
0926 06:20:58.573033 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:58.573066 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:58.573516 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:58.573711 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:58.573714 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:58.574290 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:58.574473 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:58.574478 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.575147 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.575148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:58.576005 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:58.576007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.576510 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.576511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:58.576664 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:58.577174 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:58.577184 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:58.579623 (+ 2439us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
W20260926 06:20:58.580082 13973 heartbeater.cc:644] Failed to heartbeat to 127.4.202.190:37671 (0 consecutive failures): Not authorized: Failed to ping master at 127.4.202.190:37671: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Sep 26 06:20:58 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:20:58 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:20:58.584196 13975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:58.580291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.130:49731)
0926 06:20:58.580345 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:58.580355 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:58.580394 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:58.580505 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:58.580506 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:58.580627 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:58.580668 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:58.580669 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.581267 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.581268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:58.581756 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:58.581758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.582117 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.582118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:58.582183 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:58.582551 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:58.582556 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:58.584123 (+ 1567us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":38}
W20260926 06:20:58.584255 13843 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:20:58.580346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:49731 (local address 127.4.202.190:37671)
0926 06:20:58.580402 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:58.580404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:58.580414 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:20:58.580450 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:58.580450 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:58.580459 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:58.580504 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:58.580742 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.581191 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.581852 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.582036 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.582116 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:58.584190 (+ 2074us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.130:49731: BlockingRecv error: recv got EOF from 127.4.202.130:49731 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 26 06:20:58 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:20:58 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:20:58.588521 13975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:58.584655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.130:39619)
0926 06:20:58.584696 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:58.584706 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:58.584741 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:58.584849 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:58.584850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:58.584973 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:58.584995 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:58.584996 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.585567 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.585568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:58.586154 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:58.586156 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.586486 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.586487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:58.586537 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:58.586909 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:58.586914 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:58.588452 (+ 1538us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260926 06:20:58.588598 13843 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:20:58.584714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:39619 (local address 127.4.202.190:37671)
0926 06:20:58.584752 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:58.584754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:58.584762 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:20:58.584792 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:58.584793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:58.584801 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:58.584848 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:58.585062 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.585494 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.586238 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.586420 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.586488 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:58.588534 (+ 2046us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.130:39619: BlockingRecv error: recv got EOF from 127.4.202.130:39619 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260926 06:20:58.588840 13973 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 06:20:58.656762 13976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:20:58.656937 13976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:20:58.656958 13976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:20:58.658038 13976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:20:58.658070 13976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:20:58.658095 13976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.131
I20260926 06:20:58.659451 13976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:37847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/krb5kdc/oryx_127.4.202.131.keytab
--principal=oryx/127.4.202.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.131:36097
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.202.131
--webserver_port=38041
--tserver_master_addrs=127.4.202.190:37671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.131
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:20:58.659611 13976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:20:58.659756 13976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:20:58.664489 13984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:20:58.664491 13981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:58.664564 13976 server_base.cc:1034] running on GCE node
W20260926 06:20:58.664505 13982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:20:58.664981 13976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:20:58.665201 13976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:20:58.666364 13976 hybrid_clock.cc:648] HybridClock initialized: now 1790403658666356 us; error 33 us; skew 500 ppm
Sep 26 06:20:58 dist-test-slave-pd0t krb5kdc[13109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403658, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.202.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:20:58.668735 13976 init.cc:377] Logged in from keytab as oryx/127.4.202.131@KRBTEST.COM (short username oryx)
I20260926 06:20:58.669116 13976 webserver.cc:460] Webserver started at http://127.4.202.131:38041/ using document root <none> and password file <none>
I20260926 06:20:58.669318 13976 fs_manager.cc:359] Metadata directory not provided
I20260926 06:20:58.669399 13976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:20:58.670674 13976 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:20:58.671211 13991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:58.671371 13976 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:20:58.671458 13976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-2/wal
uuid: "f8cbf0e48cdb43108d27809a6b0a64aa"
format_stamp: "Formatted at 2026-09-26 06:20:58 on dist-test-slave-pd0t"
server_key: "413870443d8d4def633c92ec8142bd25"
server_key_iv: "4671796d970bf3a35fed2f6acfe0eda7"
server_key_version: "encryptionkey@0"
I20260926 06:20:58.671753 13976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:20:58.683461 13976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:20:58.685519 13976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:20:58.685689 13976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:20:58.685883 13976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:20:58.686200 13976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:20:58.686254 13976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:58.686291 13976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:20:58.686331 13976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:20:58.691838 13976 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.131:36097
I20260926 06:20:58.691885 14104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.131:36097 every 8 connection(s)
I20260926 06:20:58.692195 13976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
Sep 26 06:20:58 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:20:58 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
W20260926 06:20:58.698539 13843 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:20:58.692776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:45335 (local address 127.4.202.190:37671)
0926 06:20:58.692837 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:58.692839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:58.693037 (+ 198us) server_negotiation.cc:407] Connection header received
0926 06:20:58.693449 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:58.693450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:58.693460 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:58.693541 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:58.694342 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.694884 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.695777 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.695962 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.696008 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:58.698479 (+ 2471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.131:45335: BlockingRecv error: recv got EOF from 127.4.202.131:45335 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 06:20:58.698607 14107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:58.692768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.131:45335)
0926 06:20:58.692904 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:58.692954 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:58.693367 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:58.693589 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:58.693592 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:58.694096 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:58.694270 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:58.694274 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.694953 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.694954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:58.695710 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:58.695712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.696021 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.696022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:58.696112 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:58.696483 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:58.696490 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:58.698420 (+ 1930us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":80,"threads_started":1}
W20260926 06:20:58.698781 14105 heartbeater.cc:644] Failed to heartbeat to 127.4.202.190:37671 (0 consecutive failures): Not authorized: Failed to ping master at 127.4.202.190:37671: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
I20260926 06:20:58.701735 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 13976
Sep 26 06:20:58 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:20:58 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:20:58.703173 14107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:58.698985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.131:37103)
0926 06:20:58.699034 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:58.699043 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:58.699081 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:58.699257 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:58.699258 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:58.699366 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:58.699405 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:58.699406 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.699972 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.699973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:58.700450 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:58.700451 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.700739 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.700740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:58.700792 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:58.701209 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:58.701232 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:58.703093 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260926 06:20:58.703204 13843 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:20:58.699124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:37103 (local address 127.4.202.190:37671)
0926 06:20:58.699168 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:58.699170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:58.699176 (+ 6us) server_negotiation.cc:407] Connection header received
0926 06:20:58.699192 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:58.699193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:58.699200 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:58.699247 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:58.699466 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.699889 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.700523 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.700685 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.700727 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:58.703170 (+ 2443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.131:37103: BlockingRecv error: recv got EOF from 127.4.202.131:37103 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Sep 26 06:20:58 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:20:58 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:20:58.707850 14107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:58.703531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.131:38417)
0926 06:20:58.703580 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:58.703590 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:58.703627 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:58.703753 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:58.703753 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:58.703845 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:58.703866 (+ 21us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:58.703867 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.704545 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.704546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:58.705317 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:58.705319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.705717 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.705718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:58.705769 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:58.706163 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:58.706168 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:58.707780 (+ 1612us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260926 06:20:58.707898 13843 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:20:58.703585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:38417 (local address 127.4.202.190:37671)
0926 06:20:58.703624 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:58.703625 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:58.703632 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:20:58.703684 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:58.703685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:58.703692 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:58.703751 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:58.703921 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.704486 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.705379 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.705651 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.705706 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:58.707842 (+ 2136us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.131:38417: BlockingRecv error: recv got EOF from 127.4.202.131:38417 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260926 06:20:58.707965 14105 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 06:20:58.709812 14108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:58.702100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41840 (local address 127.4.202.190:37671)
0926 06:20:58.702208 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:58.702210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:58.702366 (+ 156us) server_negotiation.cc:407] Connection header received
0926 06:20:58.702434 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:58.702434 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:58.702448 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:58.702509 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:58.703560 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.704260 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.705289 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:58.705548 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:58.705610 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:58.706940 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:20:58.706944 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:20:58.706945 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:20:58.706957 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:20:58.708775 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:58.709077 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:58.709080 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:58.709080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:58.709158 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:20:58.709312 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:20:58.709312 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:20:58.709312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:20:58.709495 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:20:58.709593 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 06:20:58.709681 (+ 88us) server_negotiation.cc:299] Negotiation successful
0926 06:20:58.709755 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":54,"threads_started":1}
Sep 26 06:20:59 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:20:59 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:20:59.468922 14110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:59.464280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.129:57769)
0926 06:20:59.464545 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:59.464557 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:59.464678 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:59.464865 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:59.464866 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:59.465028 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:59.465075 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:59.465077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:59.465717 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:59.465719 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:59.466321 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:59.466331 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:59.466711 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:59.466712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:59.466758 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:59.467156 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:59.467160 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:59.468857 (+ 1697us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":187,"threads_started":1}
W20260926 06:20:59.469017 14111 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:20:59.464457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:57769 (local address 127.4.202.190:37671)
0926 06:20:59.464558 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:59.464561 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:59.464656 (+ 95us) server_negotiation.cc:407] Connection header received
0926 06:20:59.464774 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:59.464774 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:59.464784 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:59.464848 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:59.465136 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:59.465633 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:59.466403 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:59.466648 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:59.466696 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:59.468948 (+ 2252us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.129:57769: BlockingRecv error: recv got EOF from 127.4.202.129:57769 (error 108)
Metrics: {"server-negotiator.queue_time_us":76,"thread_start_us":46,"threads_started":1}
Sep 26 06:20:59 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:20:59 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:20:59.593662 14112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:59.589327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.130:50491)
0926 06:20:59.589455 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:59.589466 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:59.589530 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:59.589658 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:59.589659 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:59.589808 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:59.589855 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:59.589856 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:59.590490 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:59.590491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:59.591095 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:59.591097 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:59.591412 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:59.591412 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:59.591449 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:59.591855 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:59.591859 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:59.593602 (+ 1743us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":100,"threads_started":1}
W20260926 06:20:59.593784 14111 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:20:59.589366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:50491 (local address 127.4.202.190:37671)
0926 06:20:59.589425 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:59.589427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:59.589525 (+ 98us) server_negotiation.cc:407] Connection header received
0926 06:20:59.589591 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:59.589592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:59.589602 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:59.589645 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:59.589911 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:59.590424 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:59.591163 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:59.591352 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:59.591403 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:59.593708 (+ 2305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.130:50491: BlockingRecv error: recv got EOF from 127.4.202.130:50491 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 26 06:20:59 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:20:59 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:20:59.712805 14113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:20:59.708343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.131:34783)
0926 06:20:59.708473 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 06:20:59.708483 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:20:59.708534 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:20:59.708657 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:20:59.708657 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:20:59.708806 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:20:59.708851 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:59.708852 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:59.709516 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:59.709517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:59.710041 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:20:59.710042 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:20:59.710431 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:20:59.710433 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:20:59.710501 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:59.710971 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:20:59.710975 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:20:59.712742 (+ 1767us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":78,"threads_started":1}
W20260926 06:20:59.712890 14111 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:20:59.708460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:34783 (local address 127.4.202.190:37671)
0926 06:20:59.708515 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 06:20:59.708517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:20:59.708527 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:20:59.708588 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:20:59.708588 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:20:59.708599 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:20:59.708643 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:20:59.708910 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:59.709446 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:59.710107 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:20:59.710324 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:20:59.710378 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:20:59.712824 (+ 2446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.131:34783: BlockingRecv error: recv got EOF from 127.4.202.131:34783 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 26 06:21:00 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:00 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:00.474007 14114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:00.469352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.129:56573)
0926 06:21:00.469485 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:00.469495 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:00.469545 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:00.469663 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:00.469664 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:00.469811 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:00.469861 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:00.469862 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:00.470468 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:00.470469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:00.471391 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:00.471393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:00.471747 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:00.471748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:00.471797 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:00.472231 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:00.472235 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:00.473955 (+ 1720us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
W20260926 06:21:00.474121 14115 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:00.469430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:56573 (local address 127.4.202.190:37671)
0926 06:21:00.469553 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:00.469555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:00.469563 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:21:00.469584 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:00.469585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:00.469594 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:00.469652 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:00.469919 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:00.470408 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:00.471455 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:00.471662 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:00.471713 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:00.474036 (+ 2323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.129:56573: BlockingRecv error: recv got EOF from 127.4.202.129:56573 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":71,"threads_started":1}
Sep 26 06:21:00 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:00 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:00.598680 14116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:00.594095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.130:40601)
0926 06:21:00.594262 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:00.594274 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:00.594386 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:00.594518 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:00.594519 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:00.594708 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:00.594752 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:00.594754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:00.595416 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:00.595417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:00.595942 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:00.595943 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:00.596305 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:00.596306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:00.596376 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:00.596792 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:00.596796 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:00.598571 (+ 1775us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":104,"threads_started":1}
W20260926 06:21:00.598740 14115 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:00.594175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:40601 (local address 127.4.202.190:37671)
0926 06:21:00.594250 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:00.594252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:00.594365 (+ 113us) server_negotiation.cc:407] Connection header received
0926 06:21:00.594443 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:00.594444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:00.594456 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:00.594508 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:00.594843 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:00.595333 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:00.596026 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:00.596224 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:00.596304 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:00.598677 (+ 2373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.130:40601: BlockingRecv error: recv got EOF from 127.4.202.130:40601 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 26 06:21:00 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:00 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:00.717628 14117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:00.713222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.131:38621)
0926 06:21:00.713352 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:00.713363 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:00.713418 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:00.713545 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:00.713545 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:00.713689 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:00.713734 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:00.713736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:00.714356 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:00.714357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:00.714910 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:00.714913 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:00.715282 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:00.715283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:00.715341 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:00.715851 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:00.715856 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:00.717526 (+ 1670us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":84,"threads_started":1}
W20260926 06:21:00.717672 14115 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:00.713346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:38621 (local address 127.4.202.190:37671)
0926 06:21:00.713408 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:00.713411 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:00.713419 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:21:00.713473 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:00.713474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:00.713484 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:00.713529 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:00.713788 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:00.714297 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:00.715007 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:00.715194 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:00.715272 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:00.717606 (+ 2334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.131:38621: BlockingRecv error: recv got EOF from 127.4.202.131:38621 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 26 06:21:01 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:01 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:01.479007 14118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:01.474417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.129:59709)
0926 06:21:01.474615 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:01.474629 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:01.474691 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:01.474847 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:01.474848 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:01.475019 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:01.475067 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:01.475068 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:01.475748 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:01.475749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:01.476274 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:01.476275 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:01.476615 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:01.476616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:01.476656 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:01.477075 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:01.477079 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:01.478951 (+ 1872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":99,"threads_started":1}
W20260926 06:21:01.479118 14119 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:01.474488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:59709 (local address 127.4.202.190:37671)
0926 06:21:01.474605 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:01.474607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:01.474680 (+ 73us) server_negotiation.cc:407] Connection header received
0926 06:21:01.474731 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:01.474732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:01.474759 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:01.474838 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:01.475158 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:01.475668 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:01.476356 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:01.476550 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:01.476608 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:01.479040 (+ 2432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.129:59709: BlockingRecv error: recv got EOF from 127.4.202.129:59709 (error 108)
Metrics: {"server-negotiator.queue_time_us":77,"thread_start_us":49,"threads_started":1}
Sep 26 06:21:01 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:01 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:01.603904 14120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:01.599279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.130:45029)
0926 06:21:01.599465 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:01.599479 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:01.599568 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:01.599729 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:01.599730 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:01.599906 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:01.599953 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:01.599954 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:01.600617 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:01.600618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:01.601141 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:01.601142 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:01.601515 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:01.601516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:01.601578 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:01.602120 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:01.602130 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:01.603847 (+ 1717us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":101,"threads_started":1}
W20260926 06:21:01.604043 14119 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:01.599395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:45029 (local address 127.4.202.190:37671)
0926 06:21:01.599469 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:01.599471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:01.599548 (+ 77us) server_negotiation.cc:407] Connection header received
0926 06:21:01.599637 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:01.599638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:01.599652 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:01.599722 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:01.600026 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:01.600533 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:01.601221 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:01.601424 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:01.601515 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:01.603960 (+ 2445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.130:45029: BlockingRecv error: recv got EOF from 127.4.202.130:45029 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 26 06:21:01 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:01 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:01.722882 14121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:01.718125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.131:60213)
0926 06:21:01.718284 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:01.718298 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:01.718371 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:01.718533 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:01.718534 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:01.718746 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:01.718793 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:01.718795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:01.719559 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:01.719560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:01.720187 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:01.720189 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:01.720551 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:01.720552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:01.720606 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:01.721022 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:01.721026 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:01.722817 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":100,"threads_started":1}
W20260926 06:21:01.723016 14119 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:01.718195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:60213 (local address 127.4.202.190:37671)
0926 06:21:01.718267 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:01.718269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:01.718352 (+ 83us) server_negotiation.cc:407] Connection header received
0926 06:21:01.718437 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:01.718438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:01.718452 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:01.718528 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:01.718886 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:01.719480 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:01.720268 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:01.720461 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:01.720540 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:01.722924 (+ 2384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.131:60213: BlockingRecv error: recv got EOF from 127.4.202.131:60213 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 26 06:21:02 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:02 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:02.483827 14122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:02.479433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.129:58633)
0926 06:21:02.479569 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:02.479580 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:02.479624 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:02.479756 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:02.479756 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:02.479907 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:02.479973 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:02.479974 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:02.480585 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:02.480586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:02.481102 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:02.481104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:02.481433 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:02.481434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:02.481493 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:02.482014 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:02.482019 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:02.483757 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":81,"threads_started":1}
W20260926 06:21:02.483974 14123 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:02.479485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:58633 (local address 127.4.202.190:37671)
0926 06:21:02.479602 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:02.479604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:02.479614 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:21:02.479679 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:02.479680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:02.479690 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:02.479739 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:02.480032 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:02.480520 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:02.481167 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:02.481350 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:02.481398 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:02.483870 (+ 2472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.129:58633: BlockingRecv error: recv got EOF from 127.4.202.129:58633 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":68,"threads_started":1}
Sep 26 06:21:02 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:02 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:02.608996 14124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:02.604356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.130:35587)
0926 06:21:02.604515 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:02.604526 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:02.604588 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:02.604702 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:02.604703 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:02.604860 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:02.604909 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:02.604910 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:02.605598 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:02.605600 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:02.606134 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:02.606135 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:02.606466 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:02.606467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:02.606506 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:02.607033 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:02.607037 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:02.608934 (+ 1897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":67,"threads_started":1}
W20260926 06:21:02.609107 14123 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:02.604454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:35587 (local address 127.4.202.190:37671)
0926 06:21:02.604522 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:02.604524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:02.604585 (+ 61us) server_negotiation.cc:407] Connection header received
0926 06:21:02.604634 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:02.604635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:02.604646 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:02.604691 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:02.604999 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:02.605511 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:02.606206 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:02.606403 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:02.606454 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:02.609016 (+ 2562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.130:35587: BlockingRecv error: recv got EOF from 127.4.202.130:35587 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 26 06:21:02 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:02 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:02.727900 14125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:02.723331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.131:60493)
0926 06:21:02.723492 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:02.723506 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:02.723557 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:02.723683 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:02.723684 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:02.723825 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:02.723907 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:02.723908 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:02.724728 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:02.724729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:02.725195 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:02.725196 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:02.725494 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:02.725495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:02.725530 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:02.725943 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:02.725947 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:02.727808 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":99,"threads_started":1}
W20260926 06:21:02.727942 14123 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:02.723415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:60493 (local address 127.4.202.190:37671)
0926 06:21:02.723498 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:02.723500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:02.723557 (+ 57us) server_negotiation.cc:407] Connection header received
0926 06:21:02.723609 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:02.723609 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:02.723620 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:02.723670 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:02.723976 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:02.724673 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:02.725262 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:02.725437 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:02.725487 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:02.727880 (+ 2393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.131:60493: BlockingRecv error: recv got EOF from 127.4.202.131:60493 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 26 06:21:03 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:03 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:03.488548 14126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:03.484269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.129:55149)
0926 06:21:03.484407 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:03.484416 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:03.484467 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:03.484579 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:03.484580 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:03.484720 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:03.484765 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:03.484766 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:03.485358 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:03.485359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:03.485846 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:03.485847 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:03.486178 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:03.486179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:03.486226 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:03.486673 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:03.486677 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:03.488488 (+ 1811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
W20260926 06:21:03.488710 14127 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:03.484312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:55149 (local address 127.4.202.190:37671)
0926 06:21:03.484428 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:03.484429 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:03.484455 (+ 26us) server_negotiation.cc:407] Connection header received
0926 06:21:03.484516 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:03.484517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:03.484526 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:03.484569 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:03.484816 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:03.485270 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:03.485922 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:03.486112 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:03.486162 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:03.488606 (+ 2444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.129:55149: BlockingRecv error: recv got EOF from 127.4.202.129:55149 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":66,"threads_started":1}
Sep 26 06:21:03 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:03 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:03.613492 14128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:03.609422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.130:38097)
0926 06:21:03.609559 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:03.609570 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:03.609618 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:03.609742 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:03.609743 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:03.609894 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:03.609939 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:03.609941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:03.610534 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:03.610535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:03.611063 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:03.611065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:03.611375 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:03.611376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:03.611414 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:03.611816 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:03.611820 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:03.613437 (+ 1617us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":84,"threads_started":1}
W20260926 06:21:03.613678 14127 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:03.609515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:38097 (local address 127.4.202.190:37671)
0926 06:21:03.609577 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:03.609579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:03.609615 (+ 36us) server_negotiation.cc:407] Connection header received
0926 06:21:03.609687 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:03.609687 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:03.609698 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:03.609738 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:03.609996 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:03.610471 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:03.611130 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:03.611310 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:03.611368 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:03.613588 (+ 2220us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.130:38097: BlockingRecv error: recv got EOF from 127.4.202.130:38097 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 26 06:21:03 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:03 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:03.732805 14129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:03.728401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.131:46243)
0926 06:21:03.728542 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:03.728556 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:03.728640 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:03.728778 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:03.728779 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:03.728949 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:03.728994 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:03.728996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:03.729650 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:03.729651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:03.730172 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:03.730173 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:03.730502 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:03.730503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:03.730558 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:03.730968 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:03.730972 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:03.732751 (+ 1779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":76,"threads_started":1}
W20260926 06:21:03.732965 14127 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:03.728488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:46243 (local address 127.4.202.190:37671)
0926 06:21:03.728546 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:03.728548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:03.728621 (+ 73us) server_negotiation.cc:407] Connection header received
0926 06:21:03.728700 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:03.728701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:03.728715 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:03.728763 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:03.729069 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:03.729570 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:03.730257 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:03.730440 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:03.730504 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:03.732865 (+ 2361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.131:46243: BlockingRecv error: recv got EOF from 127.4.202.131:46243 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 26 06:21:04 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:04 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:04.499567 14130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:04.489047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.129:55909)
0926 06:21:04.489200 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:04.489214 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:04.489281 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:04.489463 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:04.489464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:04.489629 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:04.489676 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:04.489677 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:04.490340 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:04.490342 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:04.490963 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:04.490966 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:04.491330 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:04.491331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:04.491372 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:04.497458 (+ 6086us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:04.497463 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:04.499499 (+ 2036us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":67,"threads_started":1}
W20260926 06:21:04.499678 14131 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:04.489113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:55909 (local address 127.4.202.190:37671)
0926 06:21:04.489266 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:04.489267 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:04.489276 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:21:04.489378 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:04.489379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:04.489389 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:04.489448 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:04.489736 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:04.490228 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:04.491049 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:04.491243 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:04.491300 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:04.499598 (+ 8298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.129:55909: BlockingRecv error: recv got EOF from 127.4.202.129:55909 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":68,"threads_started":1}
Sep 26 06:21:04 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:04 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:04.618515 14132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:04.613898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.130:46855)
0926 06:21:04.614015 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:04.614026 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:04.614087 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:04.614212 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:04.614213 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:04.614370 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:04.614416 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:04.614417 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:04.615167 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:04.615168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:04.615690 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:04.615691 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:04.616034 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:04.616035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:04.616073 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:04.616547 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:04.616552 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:04.618439 (+ 1887us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":70,"threads_started":1}
W20260926 06:21:04.618646 14131 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:04.613979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:46855 (local address 127.4.202.190:37671)
0926 06:21:04.614020 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:04.614022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:04.614072 (+ 50us) server_negotiation.cc:407] Connection header received
0926 06:21:04.614130 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:04.614131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:04.614142 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:04.614200 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:04.614477 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:04.615098 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:04.615758 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:04.615954 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:04.616037 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:04.618549 (+ 2512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.130:46855: BlockingRecv error: recv got EOF from 127.4.202.130:46855 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Sep 26 06:21:04 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:04 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:04.737721 14133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:04.733204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.131:52245)
0926 06:21:04.733318 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:04.733329 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:04.733385 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:04.733575 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:04.733576 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:04.733735 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:04.733777 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:04.733779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:04.734505 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:04.734506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:04.735079 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:04.735081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:04.735464 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:04.735465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:04.735524 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:04.735975 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:04.735979 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:04.737666 (+ 1687us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":65,"threads_started":1}
W20260926 06:21:04.737841 14131 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:04.733387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:52245 (local address 127.4.202.190:37671)
0926 06:21:04.733448 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:04.733450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:04.733457 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:21:04.733481 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:04.733482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:04.733492 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:04.733554 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:04.733861 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:04.734419 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:04.735165 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:04.735360 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:04.735443 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:04.737763 (+ 2320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.131:52245: BlockingRecv error: recv got EOF from 127.4.202.131:52245 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 26 06:21:05 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:05 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:05.504491 14134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:05.500065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.129:47977)
0926 06:21:05.500246 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:05.500258 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:05.500315 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:05.500464 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:05.500465 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:05.500623 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:05.500672 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:05.500673 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:05.501284 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:05.501285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:05.501810 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:05.501811 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:05.502148 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:05.502149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:05.502211 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:05.502712 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:05.502718 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:05.504433 (+ 1715us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":94,"threads_started":1}
W20260926 06:21:05.504626 14135 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:05.500168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:47977 (local address 127.4.202.190:37671)
0926 06:21:05.500285 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:05.500287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:05.500298 (+ 11us) server_negotiation.cc:407] Connection header received
0926 06:21:05.500370 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:05.500370 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:05.500381 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:05.500445 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:05.500732 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:05.501216 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:05.501889 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:05.502072 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:05.502147 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:05.504521 (+ 2374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.129:47977: BlockingRecv error: recv got EOF from 127.4.202.129:47977 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":51,"threads_started":1}
Sep 26 06:21:05 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:05 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:05.623775 14136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:05.619155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.130:37619)
0926 06:21:05.619320 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:05.619348 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:05.619424 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:05.619598 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:05.619599 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:05.619809 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:05.619876 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:05.619878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:05.620552 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:05.620553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:05.621093 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:05.621094 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:05.621462 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:05.621463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:05.621522 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:05.621955 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:05.621959 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:05.623682 (+ 1723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":65,"threads_started":1}
W20260926 06:21:05.623822 14135 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:05.619222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:37619 (local address 127.4.202.190:37671)
0926 06:21:05.619314 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:05.619316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:05.619469 (+ 153us) server_negotiation.cc:407] Connection header received
0926 06:21:05.619497 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:05.619498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:05.619512 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:05.619588 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:05.619951 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:05.620471 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:05.621173 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:05.621380 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:05.621461 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:05.623759 (+ 2298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.130:37619: BlockingRecv error: recv got EOF from 127.4.202.130:37619 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 26 06:21:05 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:05 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:05.742460 14137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:05.738137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.131:51503)
0926 06:21:05.738263 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:05.738272 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:05.738326 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:05.738469 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:05.738469 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:05.738668 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:05.738716 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:05.738718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:05.739302 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:05.739303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:05.739798 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:05.739799 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:05.740088 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:05.740088 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:05.740127 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:05.740587 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:05.740591 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:05.742393 (+ 1802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":70,"threads_started":1}
W20260926 06:21:05.742525 14135 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:05.738195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:51503 (local address 127.4.202.190:37671)
0926 06:21:05.738239 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:05.738241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:05.738320 (+ 79us) server_negotiation.cc:407] Connection header received
0926 06:21:05.738379 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:05.738380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:05.738392 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:05.738450 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:05.738782 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:05.739249 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:05.739865 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:05.740032 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:05.740079 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:05.742479 (+ 2400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.131:51503: BlockingRecv error: recv got EOF from 127.4.202.131:51503 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Sep 26 06:21:06 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:06 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:06.509645 14138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:06.504926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.129:33063)
0926 06:21:06.505111 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:06.505122 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:06.505174 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:06.505312 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:06.505313 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:06.505469 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:06.505517 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:06.505518 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:06.506278 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:06.506279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:06.507107 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:06.507110 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:06.507470 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:06.507470 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:06.507510 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:06.507920 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:06.507924 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:06.509579 (+ 1655us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":116,"threads_started":1}
W20260926 06:21:06.509768 14139 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:06.505076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:33063 (local address 127.4.202.190:37671)
0926 06:21:06.505198 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:06.505200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:06.505207 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:21:06.505227 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:06.505228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:06.505238 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:06.505294 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:06.505578 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:06.506087 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:06.507172 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:06.507385 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:06.507461 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:06.509666 (+ 2205us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.129:33063: BlockingRecv error: recv got EOF from 127.4.202.129:33063 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":76,"threads_started":1}
Sep 26 06:21:06 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:06 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:06.629127 14140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:06.624329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.130:37107)
0926 06:21:06.624474 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:06.624486 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:06.624554 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:06.624711 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:06.624712 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:06.624879 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:06.624944 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:06.624945 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:06.625696 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:06.625697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:06.626217 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:06.626219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:06.626543 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:06.626544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:06.626600 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:06.627066 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:06.627070 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:06.629069 (+ 1999us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":87,"threads_started":1}
W20260926 06:21:06.629262 14139 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:06.624390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:37107 (local address 127.4.202.190:37671)
0926 06:21:06.624441 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:06.624443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:06.624606 (+ 163us) server_negotiation.cc:407] Connection header received
0926 06:21:06.624638 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:06.624639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:06.624652 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:06.624699 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:06.625007 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:06.625631 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:06.626299 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:06.626484 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:06.626534 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:06.629147 (+ 2613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.130:37107: BlockingRecv error: recv got EOF from 127.4.202.130:37107 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Sep 26 06:21:06 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:06 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:06.747514 14141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:06.743177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.131:45351)
0926 06:21:06.743314 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:06.743324 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:06.743404 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:06.743582 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:06.743583 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:06.743754 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:06.743800 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:06.743802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:06.744449 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:06.744450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:06.744952 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:06.744953 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:06.745307 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:06.745307 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:06.745359 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:06.745764 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:06.745768 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:06.747456 (+ 1688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":59,"threads_started":1}
W20260926 06:21:06.747617 14139 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:06.743217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:45351 (local address 127.4.202.190:37671)
0926 06:21:06.743289 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:06.743291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:06.743387 (+ 96us) server_negotiation.cc:407] Connection header received
0926 06:21:06.743493 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:06.743494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:06.743513 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:06.743576 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:06.743883 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:06.744373 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:06.745029 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:06.745231 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:06.745308 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:06.747545 (+ 2237us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.131:45351: BlockingRecv error: recv got EOF from 127.4.202.131:45351 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 26 06:21:07 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:07 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:07.514621 14142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:07.510107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.129:38707)
0926 06:21:07.510237 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:07.510255 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:07.510310 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:07.510450 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:07.510451 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:07.510643 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:07.510690 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:07.510692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:07.511341 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:07.511342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:07.511874 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:07.511876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:07.512195 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:07.512196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:07.512236 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:07.512733 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:07.512738 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:07.514518 (+ 1780us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":75,"threads_started":1}
W20260926 06:21:07.514743 14143 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:07.510184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:38707 (local address 127.4.202.190:37671)
0926 06:21:07.510313 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:07.510315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:07.510346 (+ 31us) server_negotiation.cc:407] Connection header received
0926 06:21:07.510366 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:07.510367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:07.510377 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:07.510426 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:07.510763 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:07.511260 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:07.511948 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:07.512130 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:07.512180 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:07.514635 (+ 2455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.129:38707: BlockingRecv error: recv got EOF from 127.4.202.129:38707 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":75,"threads_started":1}
Sep 26 06:21:07 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:07 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:07.634146 14144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:07.629583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.130:35565)
0926 06:21:07.629711 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:07.629723 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:07.629783 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:07.629929 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:07.629930 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:07.630091 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:07.630136 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:07.630137 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:07.630929 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:07.630930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:07.631454 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:07.631456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:07.631792 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:07.631792 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:07.631830 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:07.632307 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:07.632312 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:07.634081 (+ 1769us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":75,"threads_started":1}
W20260926 06:21:07.634277 14143 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:07.629679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:35565 (local address 127.4.202.190:37671)
0926 06:21:07.629746 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:07.629748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:07.629767 (+ 19us) server_negotiation.cc:407] Connection header received
0926 06:21:07.629842 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:07.629843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:07.629854 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:07.629909 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:07.630197 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:07.630826 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:07.631539 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:07.631724 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:07.631781 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:07.634177 (+ 2396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.130:35565: BlockingRecv error: recv got EOF from 127.4.202.130:35565 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 26 06:21:07 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:07 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:07.752935 14145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:07.747982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.131:53957)
0926 06:21:07.748162 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:07.748174 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:07.748265 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:07.748430 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:07.748431 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:07.748602 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:07.748648 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:07.748650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:07.749374 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:07.749375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:07.749924 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:07.749926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:07.750288 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:07.750289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:07.750345 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:07.750917 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:07.750921 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:07.752834 (+ 1913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":104,"threads_started":1}
W20260926 06:21:07.752964 14143 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:07.748059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:53957 (local address 127.4.202.190:37671)
0926 06:21:07.748128 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:07.748130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:07.748245 (+ 115us) server_negotiation.cc:407] Connection header received
0926 06:21:07.748327 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:07.748328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:07.748351 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:07.748409 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:07.748735 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:07.749223 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:07.750023 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:07.750211 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:07.750293 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:07.752901 (+ 2608us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.131:53957: BlockingRecv error: recv got EOF from 127.4.202.131:53957 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Sep 26 06:21:08 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:08 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:08.520008 14146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:08.515098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.129:56637)
0926 06:21:08.515254 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:08.515266 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:08.515317 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:08.515440 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:08.515440 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:08.515672 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:08.515719 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:08.515721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:08.516599 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:08.516600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:08.517200 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:08.517201 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:08.517657 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:08.517657 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:08.517694 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:08.518114 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:08.518118 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:08.519951 (+ 1833us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":97,"threads_started":1}
W20260926 06:21:08.520159 14147 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:08.515178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:56637 (local address 127.4.202.190:37671)
0926 06:21:08.515291 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:08.515292 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:08.515305 (+ 13us) server_negotiation.cc:407] Connection header received
0926 06:21:08.515368 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:08.515369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:08.515379 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:08.515435 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:08.515823 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:08.516533 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:08.517320 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:08.517584 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:08.517650 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:08.520062 (+ 2412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.129:56637: BlockingRecv error: recv got EOF from 127.4.202.129:56637 (error 108)
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":54,"threads_started":1}
Sep 26 06:21:08 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:08 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:08.638965 14148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:08.634546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.130:35299)
0926 06:21:08.634687 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:08.634698 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:08.634760 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:08.634881 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:08.634882 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:08.635044 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:08.635091 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:08.635092 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:08.635676 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:08.635677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:08.636198 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:08.636199 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:08.636508 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:08.636509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:08.636547 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:08.636954 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:08.636959 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:08.638908 (+ 1949us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":91,"threads_started":1}
W20260926 06:21:08.639084 14147 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:08.634619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:35299 (local address 127.4.202.190:37671)
0926 06:21:08.634664 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:08.634666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:08.634763 (+ 97us) server_negotiation.cc:407] Connection header received
0926 06:21:08.634823 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:08.634824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:08.634834 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:08.634871 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:08.635146 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:08.635610 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:08.636267 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:08.636448 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:08.636496 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:08.639002 (+ 2506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.130:35299: BlockingRecv error: recv got EOF from 127.4.202.130:35299 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Sep 26 06:21:08 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:08 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:08.757599 14149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:08.753370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.131:60653)
0926 06:21:08.753503 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:08.753513 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:08.753565 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:08.753683 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:08.753683 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:08.753829 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:08.753879 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:08.753880 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:08.754483 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:08.754484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:08.755034 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:08.755036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:08.755345 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:08.755346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:08.755383 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:08.755803 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:08.755807 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:08.757506 (+ 1699us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":75,"threads_started":1}
W20260926 06:21:08.757639 14147 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:08.753479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:60653 (local address 127.4.202.190:37671)
0926 06:21:08.753523 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:08.753524 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:08.753551 (+ 27us) server_negotiation.cc:407] Connection header received
0926 06:21:08.753617 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:08.753617 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:08.753628 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:08.753678 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:08.753934 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:08.754414 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:08.755104 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:08.755277 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:08.755335 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:08.757579 (+ 2244us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.131:60653: BlockingRecv error: recv got EOF from 127.4.202.131:60653 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Sep 26 06:21:09 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:09 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:09.530527 14150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:09.520465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.129:54023)
0926 06:21:09.520600 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:09.520615 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:09.520665 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:09.520794 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:09.520794 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:09.520961 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:09.521004 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:09.521005 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:09.521687 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:09.521688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:09.522230 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:09.522232 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:09.522550 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:09.522551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:09.522658 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:09.528607 (+ 5949us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:09.528613 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:09.530411 (+ 1798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":83,"threads_started":1}
W20260926 06:21:09.530567 14151 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:09.520521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:54023 (local address 127.4.202.190:37671)
0926 06:21:09.520665 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:09.520667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:09.520676 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:21:09.520701 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:09.520702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:09.520712 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:09.520776 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:09.521099 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:09.521617 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:09.522299 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:09.522486 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:09.522535 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:09.530492 (+ 7957us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.129:54023: BlockingRecv error: recv got EOF from 127.4.202.129:54023 (error 108)
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":89,"threads_started":1}
Sep 26 06:21:09 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:09 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:09.643756 14152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:09.639423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.130:54073)
0926 06:21:09.639582 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:09.639592 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:09.639653 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:09.639777 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:09.639778 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:09.639934 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:09.639978 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:09.639979 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:09.640560 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:09.640561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:09.641071 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:09.641072 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:09.641517 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:09.641519 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:09.641565 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:09.641983 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:09.641987 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:09.643690 (+ 1703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":87,"threads_started":1}
W20260926 06:21:09.643879 14151 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:09.639521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:54073 (local address 127.4.202.190:37671)
0926 06:21:09.639586 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:09.639588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:09.639647 (+ 59us) server_negotiation.cc:407] Connection header received
0926 06:21:09.639699 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:09.639700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:09.639710 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:09.639756 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:09.640031 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:09.640499 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:09.641152 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:09.641442 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:09.641503 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:09.643776 (+ 2273us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.130:54073: BlockingRecv error: recv got EOF from 127.4.202.130:54073 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 26 06:21:09 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:09 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:09.762789 14153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:09.758120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.131:58077)
0926 06:21:09.758300 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:09.758309 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:09.758359 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:09.758483 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:09.758484 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:09.758716 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:09.758784 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:09.758787 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:09.759594 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:09.759594 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:09.760126 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:09.760127 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:09.760439 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:09.760440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:09.760488 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:09.760961 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:09.760966 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:09.762726 (+ 1760us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":108,"threads_started":1}
W20260926 06:21:09.762890 14151 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:09.758208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:58077 (local address 127.4.202.190:37671)
0926 06:21:09.758262 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:09.758264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:09.758361 (+ 97us) server_negotiation.cc:407] Connection header received
0926 06:21:09.758400 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:09.758400 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:09.758412 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:09.758465 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:09.758875 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:09.759527 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:09.760194 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:09.760377 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:09.760429 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:09.762819 (+ 2390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.131:58077: BlockingRecv error: recv got EOF from 127.4.202.131:58077 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Sep 26 06:21:10 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:10 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:10.535562 14154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:10.531024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.129:35135)
0926 06:21:10.531149 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:10.531168 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:10.531213 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:10.531378 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:10.531379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:10.531537 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:10.531583 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:10.531584 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:10.532201 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:10.532201 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:10.532772 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:10.532773 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:10.533090 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:10.533091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:10.533134 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:10.533554 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:10.533559 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:10.535503 (+ 1944us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":74,"threads_started":1}
W20260926 06:21:10.535725 14155 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:10.531093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:35135 (local address 127.4.202.190:37671)
0926 06:21:10.531204 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:10.531206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:10.531215 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:21:10.531271 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:10.531272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:10.531282 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:10.531337 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:10.531642 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:10.532128 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:10.532840 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:10.533027 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:10.533078 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:10.535641 (+ 2563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.129:35135: BlockingRecv error: recv got EOF from 127.4.202.129:35135 (error 108)
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":65,"threads_started":1}
Sep 26 06:21:10 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:10 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:10.648638 14156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:10.644227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.130:56921)
0926 06:21:10.644396 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:10.644406 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:10.644458 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:10.644586 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:10.644586 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:10.644743 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:10.644791 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:10.644792 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:10.645408 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:10.645409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:10.645924 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:10.645926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:10.646230 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:10.646231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:10.646269 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:10.646755 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:10.646760 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:10.648546 (+ 1786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":88,"threads_started":1}
W20260926 06:21:10.648670 14155 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:10.644330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:56921 (local address 127.4.202.190:37671)
0926 06:21:10.644394 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:10.644396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:10.644446 (+ 50us) server_negotiation.cc:407] Connection header received
0926 06:21:10.644514 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:10.644515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:10.644525 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:10.644567 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:10.644862 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:10.645334 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:10.645994 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:10.646169 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:10.646218 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:10.648613 (+ 2395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.130:56921: BlockingRecv error: recv got EOF from 127.4.202.130:56921 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 26 06:21:10 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:10 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:10.767691 14157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:10.763224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.131:33783)
0926 06:21:10.763409 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:10.763422 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:10.763493 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:10.763656 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:10.763657 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:10.763837 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:10.763880 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:10.763882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:10.764505 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:10.764506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:10.765012 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:10.765014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:10.765380 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:10.765381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:10.765453 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:10.765920 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:10.765925 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:10.767603 (+ 1678us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":89,"threads_started":1}
W20260926 06:21:10.767727 14155 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:10.763364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:33783 (local address 127.4.202.190:37671)
0926 06:21:10.763425 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:10.763427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:10.763475 (+ 48us) server_negotiation.cc:407] Connection header received
0926 06:21:10.763559 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:10.763560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:10.763579 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:10.763636 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:10.763955 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:10.764425 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:10.765095 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:10.765277 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:10.765380 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:10.767669 (+ 2289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.131:33783: BlockingRecv error: recv got EOF from 127.4.202.131:33783 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 26 06:21:11 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:11 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:11.540624 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:11.536017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.129:55625)
0926 06:21:11.536146 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:11.536158 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:11.536212 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:11.536336 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:11.536337 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:11.536495 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:11.536560 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:11.536561 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:11.537206 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:11.537208 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:11.537751 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:11.537753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:11.538094 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:11.538095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:11.538135 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:11.538660 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:11.538665 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:11.540567 (+ 1902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":80,"threads_started":1}
W20260926 06:21:11.540735 14159 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:11.536091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:55625 (local address 127.4.202.190:37671)
0926 06:21:11.536214 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:11.536218 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:11.536229 (+ 11us) server_negotiation.cc:407] Connection header received
0926 06:21:11.536249 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:11.536250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:11.536260 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:11.536317 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:11.536628 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:11.537112 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:11.537834 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:11.538027 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:11.538081 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:11.540651 (+ 2570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.129:55625: BlockingRecv error: recv got EOF from 127.4.202.129:55625 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":76,"threads_started":1}
Sep 26 06:21:11 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:11 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:11.653828 14160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:11.649111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.130:36331)
0926 06:21:11.649247 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:11.649260 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:11.649351 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:11.649509 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:11.649510 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:11.649691 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:11.649737 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:11.649739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:11.650422 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:11.650423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:11.651017 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:11.651020 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:11.651391 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:11.651393 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:11.651450 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:11.651876 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:11.651880 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:11.653724 (+ 1844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":62,"threads_started":1}
W20260926 06:21:11.653887 14159 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:11.649266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:36331 (local address 127.4.202.190:37671)
0926 06:21:11.649340 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:11.649342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:11.649352 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:21:11.649413 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:11.649414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:11.649428 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:11.649504 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:11.649819 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:11.650331 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:11.651103 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:11.651296 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:11.651390 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:11.653807 (+ 2417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.130:36331: BlockingRecv error: recv got EOF from 127.4.202.130:36331 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 26 06:21:11 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:11 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:11.772689 14161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:11.768242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.131:48641)
0926 06:21:11.768404 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:11.768418 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:11.768512 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:11.768665 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:11.768665 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:11.768838 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:11.768881 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:11.768883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:11.769549 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:11.769550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:11.770057 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:11.770059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:11.770450 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:11.770450 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:11.770513 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:11.770944 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:11.770948 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:11.772635 (+ 1687us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":81,"threads_started":1}
W20260926 06:21:11.772815 14159 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:11.768328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:48641 (local address 127.4.202.190:37671)
0926 06:21:11.768409 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:11.768412 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:11.768493 (+ 81us) server_negotiation.cc:407] Connection header received
0926 06:21:11.768597 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:11.768598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:11.768611 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:11.768661 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:11.768964 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:11.769467 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:11.770145 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:11.770330 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:11.770440 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:11.772738 (+ 2298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.131:48641: BlockingRecv error: recv got EOF from 127.4.202.131:48641 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 26 06:21:12 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:12 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:12.545663 14162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:12.541136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.129:43237)
0926 06:21:12.541261 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:12.541274 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:12.541338 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:12.541510 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:12.541510 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:12.541668 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:12.541721 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:12.541722 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:12.542340 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:12.542341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:12.542925 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:12.542927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:12.543242 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:12.543242 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:12.543279 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:12.543779 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:12.543784 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:12.545561 (+ 1777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":63,"threads_started":1}
W20260926 06:21:12.545708 14163 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:12.541204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:43237 (local address 127.4.202.190:37671)
0926 06:21:12.541360 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:12.541363 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:12.541373 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:21:12.541414 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:12.541415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:12.541424 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:12.541483 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:12.541783 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:12.542274 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:12.542991 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:12.543178 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:12.543230 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:12.545636 (+ 2406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.129:43237: BlockingRecv error: recv got EOF from 127.4.202.129:43237 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":85,"threads_started":1}
Sep 26 06:21:12 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:12 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:12.658951 14164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:12.654314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.130:35617)
0926 06:21:12.654475 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:12.654487 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:12.654553 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:12.654784 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:12.654785 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:12.655056 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:12.655112 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:12.655115 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:12.655771 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:12.655771 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:12.656300 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:12.656302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:12.656632 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:12.656633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:12.656670 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:12.657096 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:12.657101 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:12.658894 (+ 1793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":88,"threads_started":1}
W20260926 06:21:12.659051 14163 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:12.654390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:35617 (local address 127.4.202.190:37671)
0926 06:21:12.654460 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:12.654462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:12.654535 (+ 73us) server_negotiation.cc:407] Connection header received
0926 06:21:12.654671 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:12.654672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:12.654685 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:12.654762 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:12.655188 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:12.655684 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:12.656388 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:12.656570 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:12.656623 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:12.658982 (+ 2359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.130:35617: BlockingRecv error: recv got EOF from 127.4.202.130:35617 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 26 06:21:12 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:12 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.131@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:12.777557 14165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:12.773125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.131:35639)
0926 06:21:12.773267 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:12.773278 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:12.773331 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:12.773436 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:12.773436 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:12.773585 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:12.773627 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:12.773628 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:12.774282 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:12.774283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:12.774952 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:12.774955 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:12.775276 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:12.775277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:12.775318 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:12.775818 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:12.775823 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:12.777494 (+ 1671us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":81,"threads_started":1}
W20260926 06:21:12.777679 14163 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:12.773167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:35639 (local address 127.4.202.190:37671)
0926 06:21:12.773214 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:12.773216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:12.773322 (+ 106us) server_negotiation.cc:407] Connection header received
0926 06:21:12.773374 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:12.773374 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:12.773386 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:12.773432 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:12.773719 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:12.774173 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:12.775036 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:12.775213 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:12.775267 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:12.777578 (+ 2311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.131:35639: BlockingRecv error: recv got EOF from 127.4.202.131:35639 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Sep 26 06:21:13 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:13 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.129@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:13.550938 14166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:13.546157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.129:45911)
0926 06:21:13.546302 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:13.546314 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:13.546364 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:13.546576 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:13.546610 (+ 34us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:13.546780 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:13.546830 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:13.546832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:13.547501 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:13.547503 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:13.548022 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:13.548024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:13.548374 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:13.548375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:13.548436 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:13.548940 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:13.548945 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:13.550839 (+ 1894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":66,"threads_started":1}
W20260926 06:21:13.550978 14167 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:13.546200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:45911 (local address 127.4.202.190:37671)
0926 06:21:13.546338 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:13.546340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:13.546366 (+ 26us) server_negotiation.cc:407] Connection header received
0926 06:21:13.546473 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:13.546474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:13.546488 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:13.546552 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:13.546899 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:13.547404 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:13.548103 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:13.548291 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:13.548394 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:13.550911 (+ 2517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.129:45911: BlockingRecv error: recv got EOF from 127.4.202.129:45911 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":85,"threads_started":1}
Sep 26 06:21:13 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 06:21:13 dist-test-slave-pd0t krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.202.130@KRBTEST.COM for oryx/127.4.202.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 06:21:13.663821 14168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:13.659433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37671 (local address 127.4.202.130:50643)
0926 06:21:13.659566 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:13.659576 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:13.659638 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:13.659764 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:13.659764 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:13.659919 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:13.659966 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:13.659968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:13.660612 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:13.660613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:13.661149 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:13.661151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:13.661481 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:13.661482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:13.661524 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:13.661949 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:13.661953 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:13.663700 (+ 1747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.202.190:37671: Server oryx/127.4.202.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
W20260926 06:21:13.663935 14167 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:13.659523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:50643 (local address 127.4.202.190:37671)
0926 06:21:13.659579 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:13.659582 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:13.659639 (+ 57us) server_negotiation.cc:407] Connection header received
0926 06:21:13.659684 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:13.659685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:13.659695 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:13.659746 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:13.660028 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:13.660520 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:13.661223 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:13.661415 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:13.661463 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:13.663860 (+ 2397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.202.130:50643: BlockingRecv error: recv got EOF from 127.4.202.130:50643 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 06:21:13.706570 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 13692
I20260926 06:21:13.711151 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 13844
I20260926 06:21:13.715323 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 13976
I20260926 06:21:13.719504 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 13622
2026-09-26T06:21:13Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17373 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-26T06:21:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:21:13Z Disabled control of system clock
I20260926 06:21:13.746031 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:44003
--webserver_interface=127.4.202.190
--webserver_port=0
--builtin_ntp_servers=127.4.202.148:38577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:44003
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260926 06:21:13.823458 14172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:21:13.823634 14172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:21:13.823652 14172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:21:13.824813 14172 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:21:13.824852 14172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:21:13.824865 14172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:21:13.824877 14172 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:21:13.824928 14172 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:21:13.826232 14172 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:38577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:44003
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:44003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:21:13.826493 14172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:21:13.826798 14172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:21:13.831918 14180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:21:13.832027 14177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:21:13.832043 14178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:21:13.832063 14172 server_base.cc:1034] running on GCE node
I20260926 06:21:13.832424 14172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:21:13.832722 14172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:21:13.833871 14172 hybrid_clock.cc:648] HybridClock initialized: now 1790403673833842 us; error 37 us; skew 500 ppm
I20260926 06:21:13.835086 14172 webserver.cc:460] Webserver started at http://127.4.202.190:41131/ using document root <none> and password file <none>
I20260926 06:21:13.835330 14172 fs_manager.cc:359] Metadata directory not provided
I20260926 06:21:13.835377 14172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:21:13.835536 14172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:21:13.836483 14172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "ca55628b541a4629a935eb53f71be99c"
format_stamp: "Formatted at 2026-09-26 06:21:13 on dist-test-slave-pd0t"
server_key: "7c9c3256c3793497922919123d168084"
server_key_iv: "20ce01378061ee9ba5fa56379fbe7e59"
server_key_version: "encryptionkey@0"
I20260926 06:21:13.836798 14172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "ca55628b541a4629a935eb53f71be99c"
format_stamp: "Formatted at 2026-09-26 06:21:13 on dist-test-slave-pd0t"
server_key: "7c9c3256c3793497922919123d168084"
server_key_iv: "20ce01378061ee9ba5fa56379fbe7e59"
server_key_version: "encryptionkey@0"
I20260926 06:21:13.837968 14172 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 06:21:13.838711 14186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:13.838886 14172 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 06:21:13.838953 14172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "ca55628b541a4629a935eb53f71be99c"
format_stamp: "Formatted at 2026-09-26 06:21:13 on dist-test-slave-pd0t"
server_key: "7c9c3256c3793497922919123d168084"
server_key_iv: "20ce01378061ee9ba5fa56379fbe7e59"
server_key_version: "encryptionkey@0"
I20260926 06:21:13.839035 14172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:21:13.850066 14172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:21:13.850302 14172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:21:13.850468 14172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:21:13.854050 14172 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:44003
I20260926 06:21:13.854095 14238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:44003 every 8 connection(s)
I20260926 06:21:13.854370 14172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:21:13.854986 14239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:21:13.857331 14239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca55628b541a4629a935eb53f71be99c: Bootstrap starting.
I20260926 06:21:13.857895 14239 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca55628b541a4629a935eb53f71be99c: Neither blocks nor log segments found. Creating new log.
I20260926 06:21:13.858178 14239 log.cc:824] T 00000000000000000000000000000000 P ca55628b541a4629a935eb53f71be99c: Log is configured to *not* fsync() on all Append() calls
I20260926 06:21:13.858872 14239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca55628b541a4629a935eb53f71be99c: No bootstrap required, opened a new log
I20260926 06:21:13.859916 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 14172
I20260926 06:21:13.860004 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:21:13.860172 4906 external_mini_cluster.cc:1428] Setting key 56b6187ce9531ebdb8033338173caaae
I20260926 06:21:13.860517 14239 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca55628b541a4629a935eb53f71be99c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca55628b541a4629a935eb53f71be99c" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 44003 } }
I20260926 06:21:13.860703 14239 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca55628b541a4629a935eb53f71be99c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:21:13.860802 14239 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca55628b541a4629a935eb53f71be99c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca55628b541a4629a935eb53f71be99c, State: Initialized, Role: FOLLOWER
I20260926 06:21:13.860988 14239 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca55628b541a4629a935eb53f71be99c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca55628b541a4629a935eb53f71be99c" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 44003 } }
I20260926 06:21:13.861064 14239 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca55628b541a4629a935eb53f71be99c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:21:13.861112 14239 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca55628b541a4629a935eb53f71be99c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:21:13.861158 14239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca55628b541a4629a935eb53f71be99c [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:21:13.861688 14239 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca55628b541a4629a935eb53f71be99c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca55628b541a4629a935eb53f71be99c" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 44003 } }
I20260926 06:21:13.861769 14239 leader_election.cc:302] T 00000000000000000000000000000000 P ca55628b541a4629a935eb53f71be99c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca55628b541a4629a935eb53f71be99c; no voters:
I20260926 06:21:13.861946 14239 leader_election.cc:288] T 00000000000000000000000000000000 P ca55628b541a4629a935eb53f71be99c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:21:13.861979 14244 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca55628b541a4629a935eb53f71be99c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:21:13.862170 14244 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca55628b541a4629a935eb53f71be99c [term 1 LEADER]: Becoming Leader. State: Replica: ca55628b541a4629a935eb53f71be99c, State: Running, Role: LEADER
I20260926 06:21:13.862253 14239 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca55628b541a4629a935eb53f71be99c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:21:13.862303 14244 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca55628b541a4629a935eb53f71be99c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca55628b541a4629a935eb53f71be99c" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 44003 } }
I20260926 06:21:13.862682 14244 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca55628b541a4629a935eb53f71be99c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca55628b541a4629a935eb53f71be99c. Latest consensus state: current_term: 1 leader_uuid: "ca55628b541a4629a935eb53f71be99c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca55628b541a4629a935eb53f71be99c" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 44003 } } }
I20260926 06:21:13.863037 14244 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca55628b541a4629a935eb53f71be99c [sys.catalog]: This master's current role is: LEADER
I20260926 06:21:13.862969 14245 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca55628b541a4629a935eb53f71be99c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca55628b541a4629a935eb53f71be99c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca55628b541a4629a935eb53f71be99c" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 44003 } } }
I20260926 06:21:13.863372 14245 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca55628b541a4629a935eb53f71be99c [sys.catalog]: This master's current role is: LEADER
I20260926 06:21:13.863473 14253 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:21:13.864009 14253 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:21:13.865424 14253 catalog_manager.cc:1347] Generated new cluster ID: 6c4e5a51bce94129925e672f58332066
I20260926 06:21:13.865474 14253 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:21:13.867282 14243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:13.860677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40700 (local address 127.4.202.190:44003)
0926 06:21:13.862149 (+ 1472us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:13.862157 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:13.862187 (+ 30us) server_negotiation.cc:407] Connection header received
0926 06:21:13.862294 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:13.862301 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:13.862367 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:13.862512 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:13.862903 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:13.864225 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:13.865247 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:13.865775 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:13.865910 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:13.866558 (+ 648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:21:13.866562 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:21:13.866562 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:21:13.866817 (+ 255us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:21:13.866852 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:21:13.866862 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:21:13.866960 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 06:21:13.867036 (+ 76us) server_negotiation.cc:299] Negotiation successful
0926 06:21:13.867118 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1399,"thread_start_us":82,"threads_started":1}
I20260926 06:21:13.880026 14253 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:21:13.880509 14253 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:21:13.882525 14253 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca55628b541a4629a935eb53f71be99c: Generated new TSK 0
I20260926 06:21:13.882705 14253 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:21:13.890849 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.129:0
--local_ip_for_outbound_sockets=127.4.202.129
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:44003
--builtin_ntp_servers=127.4.202.148:38577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260926 06:21:13.985766 14263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:21:13.985987 14263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:21:13.986019 14263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:21:13.987118 14263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:21:13.987176 14263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:21:13.987216 14263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.129
I20260926 06:21:13.988507 14263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:38577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:44003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.129
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:21:13.988759 14263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:21:13.989041 14263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:21:13.994709 14268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:21:13.994725 14269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:21:13.994786 14271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:21:13.995177 14263 server_base.cc:1034] running on GCE node
I20260926 06:21:13.995393 14263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:21:13.995667 14263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:21:13.996834 14263 hybrid_clock.cc:648] HybridClock initialized: now 1790403673996817 us; error 34 us; skew 500 ppm
I20260926 06:21:13.998076 14263 webserver.cc:460] Webserver started at http://127.4.202.129:41629/ using document root <none> and password file <none>
I20260926 06:21:13.998327 14263 fs_manager.cc:359] Metadata directory not provided
I20260926 06:21:13.998397 14263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:21:13.998617 14263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:21:13.999585 14263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/data/instance:
uuid: "7847b7417d4d4bdfaabb49c9f80f963d"
format_stamp: "Formatted at 2026-09-26 06:21:13 on dist-test-slave-pd0t"
server_key: "1d02840511592eba4a7fbcc50ec2aca1"
server_key_iv: "78536da88456e01c9b465bf7cdaae29f"
server_key_version: "encryptionkey@0"
I20260926 06:21:13.999943 14263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance:
uuid: "7847b7417d4d4bdfaabb49c9f80f963d"
format_stamp: "Formatted at 2026-09-26 06:21:13 on dist-test-slave-pd0t"
server_key: "1d02840511592eba4a7fbcc50ec2aca1"
server_key_iv: "78536da88456e01c9b465bf7cdaae29f"
server_key_version: "encryptionkey@0"
I20260926 06:21:14.001212 14263 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:21:14.001922 14277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:14.002096 14263 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:21:14.002188 14263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/wal
uuid: "7847b7417d4d4bdfaabb49c9f80f963d"
format_stamp: "Formatted at 2026-09-26 06:21:13 on dist-test-slave-pd0t"
server_key: "1d02840511592eba4a7fbcc50ec2aca1"
server_key_iv: "78536da88456e01c9b465bf7cdaae29f"
server_key_version: "encryptionkey@0"
I20260926 06:21:14.002265 14263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:21:14.024456 14263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:21:14.024744 14263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:21:14.024907 14263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:21:14.025125 14263 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:21:14.025460 14263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:21:14.025512 14263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:14.025552 14263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:21:14.025578 14263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:14.031339 14263 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.129:35225
I20260926 06:21:14.031436 14390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.129:35225 every 8 connection(s)
I20260926 06:21:14.031734 14263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
I20260926 06:21:14.036012 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 14263
I20260926 06:21:14.036080 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance
I20260926 06:21:14.036266 4906 external_mini_cluster.cc:1428] Setting key 3728ae2f3b730490605596ef24e8868b
I20260926 06:21:14.036945 14243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:14.032466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:42797 (local address 127.4.202.190:44003)
0926 06:21:14.032530 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:14.032532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:14.032773 (+ 241us) server_negotiation.cc:407] Connection header received
0926 06:21:14.033299 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:14.033301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:14.033314 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:14.033363 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:14.033991 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:14.034493 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:14.035527 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:14.035745 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:14.035830 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:14.036646 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:21:14.036648 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:21:14.036648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:21:14.036651 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:21:14.036666 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:21:14.036670 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:21:14.036723 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 06:21:14.036837 (+ 114us) server_negotiation.cc:299] Negotiation successful
0926 06:21:14.036864 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 06:21:14.037109 14393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:14.032450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:44003 (local address 127.4.202.129:42797)
0926 06:21:14.032662 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:14.032693 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:14.033195 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:14.033427 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:14.033431 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:14.033438 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:14.033897 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:14.033901 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:14.034626 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:14.034627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:14.035434 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:14.035436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:14.035841 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:14.035842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:14.035969 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:14.036536 (+ 567us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:21:14.036536 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:14.036563 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:21:14.036583 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:21:14.036732 (+ 149us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:21:14.036735 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:21:14.036737 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:21:14.036790 (+ 53us) client_negotiation.cc:240] Negotiation successful
0926 06:21:14.036865 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":103,"threads_started":1}
I20260926 06:21:14.037196 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.130:0
--local_ip_for_outbound_sockets=127.4.202.130
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:44003
--builtin_ntp_servers=127.4.202.148:38577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260926 06:21:14.037572 14391 heartbeater.cc:347] Connected to a master server at 127.4.202.190:44003
I20260926 06:21:14.037647 14391 heartbeater.cc:464] Registering TS with master...
I20260926 06:21:14.037820 14391 heartbeater.cc:511] Master 127.4.202.190:44003 requested a full tablet report, sending...
I20260926 06:21:14.038314 14203 ts_manager.cc:194] Registered new tserver with Master: 7847b7417d4d4bdfaabb49c9f80f963d (127.4.202.129:35225)
I20260926 06:21:14.038992 14203 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.202.129:42797
W20260926 06:21:14.115243 14394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:21:14.115433 14394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:21:14.115453 14394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:21:14.116496 14394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:21:14.116528 14394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:21:14.116551 14394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.130
I20260926 06:21:14.117741 14394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:38577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:44003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.130
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:21:14.117911 14394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:21:14.118158 14394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:21:14.122992 14399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:21:14.123042 14400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:21:14.122993 14402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:21:14.123431 14394 server_base.cc:1034] running on GCE node
I20260926 06:21:14.123658 14394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:21:14.123924 14394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:21:14.125080 14394 hybrid_clock.cc:648] HybridClock initialized: now 1790403674125069 us; error 25 us; skew 500 ppm
I20260926 06:21:14.126263 14394 webserver.cc:460] Webserver started at http://127.4.202.130:43783/ using document root <none> and password file <none>
I20260926 06:21:14.126503 14394 fs_manager.cc:359] Metadata directory not provided
I20260926 06:21:14.126546 14394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:21:14.126713 14394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:21:14.127733 14394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/data/instance:
uuid: "661215cd92e545a39310bfa3bb98c943"
format_stamp: "Formatted at 2026-09-26 06:21:14 on dist-test-slave-pd0t"
server_key: "eaae52bb3f7d011675771ff4796d8ca3"
server_key_iv: "e5ee7dd79764b86de244a46d9ad12c59"
server_key_version: "encryptionkey@0"
I20260926 06:21:14.128099 14394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance:
uuid: "661215cd92e545a39310bfa3bb98c943"
format_stamp: "Formatted at 2026-09-26 06:21:14 on dist-test-slave-pd0t"
server_key: "eaae52bb3f7d011675771ff4796d8ca3"
server_key_iv: "e5ee7dd79764b86de244a46d9ad12c59"
server_key_version: "encryptionkey@0"
I20260926 06:21:14.129374 14394 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:21:14.130045 14408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:14.130227 14394 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:21:14.130319 14394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/wal
uuid: "661215cd92e545a39310bfa3bb98c943"
format_stamp: "Formatted at 2026-09-26 06:21:14 on dist-test-slave-pd0t"
server_key: "eaae52bb3f7d011675771ff4796d8ca3"
server_key_iv: "e5ee7dd79764b86de244a46d9ad12c59"
server_key_version: "encryptionkey@0"
I20260926 06:21:14.130404 14394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:21:14.147286 14394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:21:14.147540 14394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:21:14.147678 14394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:21:14.147881 14394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:21:14.148168 14394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:21:14.148221 14394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:14.148259 14394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:21:14.148288 14394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:14.153774 14394 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.130:35279
I20260926 06:21:14.153820 14521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.130:35279 every 8 connection(s)
I20260926 06:21:14.154096 14394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
I20260926 06:21:14.158711 14243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:14.154737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:38261 (local address 127.4.202.190:44003)
0926 06:21:14.154783 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:14.154785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:14.155004 (+ 219us) server_negotiation.cc:407] Connection header received
0926 06:21:14.155415 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:14.155416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:14.155425 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:14.155477 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:14.156079 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:14.156548 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:14.157495 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:14.157692 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:14.157754 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:14.158452 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:21:14.158453 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:21:14.158453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:21:14.158457 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:21:14.158472 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:21:14.158476 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:21:14.158518 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0926 06:21:14.158618 (+ 100us) server_negotiation.cc:299] Negotiation successful
0926 06:21:14.158646 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 06:21:14.158877 14524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:14.154716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:44003 (local address 127.4.202.130:38261)
0926 06:21:14.154888 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:14.154931 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:14.155325 (+ 394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:14.155541 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:14.155544 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:14.155552 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:14.156011 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:14.156015 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:14.156622 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:14.156623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:14.157387 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:14.157389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:14.157770 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:14.157770 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:14.157881 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:14.158305 (+ 424us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:21:14.158306 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:14.158330 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:21:14.158352 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:21:14.158518 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:21:14.158520 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:21:14.158522 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:21:14.158568 (+ 46us) client_negotiation.cc:240] Negotiation successful
0926 06:21:14.158689 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":80,"threads_started":1}
I20260926 06:21:14.159322 14522 heartbeater.cc:347] Connected to a master server at 127.4.202.190:44003
I20260926 06:21:14.159400 14522 heartbeater.cc:464] Registering TS with master...
I20260926 06:21:14.159579 14522 heartbeater.cc:511] Master 127.4.202.190:44003 requested a full tablet report, sending...
I20260926 06:21:14.159946 14203 ts_manager.cc:194] Registered new tserver with Master: 661215cd92e545a39310bfa3bb98c943 (127.4.202.130:35279)
I20260926 06:21:14.160341 14203 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.202.130:38261
I20260926 06:21:14.162189 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 14394
I20260926 06:21:14.162271 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance
I20260926 06:21:14.162425 4906 external_mini_cluster.cc:1428] Setting key c084789115572b3c5f5d35de5347a689
I20260926 06:21:14.163226 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.131:0
--local_ip_for_outbound_sockets=127.4.202.131
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:44003
--builtin_ntp_servers=127.4.202.148:38577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260926 06:21:14.240298 14525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:21:14.240511 14525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:21:14.240546 14525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:21:14.241629 14525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:21:14.241698 14525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:21:14.241770 14525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.131
I20260926 06:21:14.243058 14525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:38577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:44003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.131
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:21:14.243227 14525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:21:14.243444 14525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:21:14.248206 14530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:21:14.248355 14531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:21:14.248207 14525 server_base.cc:1034] running on GCE node
W20260926 06:21:14.248373 14533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:21:14.248783 14525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:21:14.249023 14525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:21:14.250172 14525 hybrid_clock.cc:648] HybridClock initialized: now 1790403674250175 us; error 38 us; skew 500 ppm
I20260926 06:21:14.251330 14525 webserver.cc:460] Webserver started at http://127.4.202.131:32927/ using document root <none> and password file <none>
I20260926 06:21:14.251578 14525 fs_manager.cc:359] Metadata directory not provided
I20260926 06:21:14.251649 14525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:21:14.251801 14525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:21:14.252795 14525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/data/instance:
uuid: "c99637c2286a4b1d9cf9bcbdc9d7589c"
format_stamp: "Formatted at 2026-09-26 06:21:14 on dist-test-slave-pd0t"
server_key: "7fcd26d5a48b31b2a3dc6a1818bf199d"
server_key_iv: "3cfcc3f60f012eef8b96c6576274c62f"
server_key_version: "encryptionkey@0"
I20260926 06:21:14.253135 14525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance:
uuid: "c99637c2286a4b1d9cf9bcbdc9d7589c"
format_stamp: "Formatted at 2026-09-26 06:21:14 on dist-test-slave-pd0t"
server_key: "7fcd26d5a48b31b2a3dc6a1818bf199d"
server_key_iv: "3cfcc3f60f012eef8b96c6576274c62f"
server_key_version: "encryptionkey@0"
I20260926 06:21:14.254338 14525 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:21:14.255137 14539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:14.255291 14525 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:14.255400 14525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/wal
uuid: "c99637c2286a4b1d9cf9bcbdc9d7589c"
format_stamp: "Formatted at 2026-09-26 06:21:14 on dist-test-slave-pd0t"
server_key: "7fcd26d5a48b31b2a3dc6a1818bf199d"
server_key_iv: "3cfcc3f60f012eef8b96c6576274c62f"
server_key_version: "encryptionkey@0"
I20260926 06:21:14.255474 14525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:21:14.263505 14525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:21:14.263803 14525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:21:14.263940 14525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:21:14.264132 14525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:21:14.264417 14525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:21:14.264467 14525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:14.264504 14525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:21:14.264531 14525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:14.270031 14525 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.131:46359
I20260926 06:21:14.270076 14652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.131:46359 every 8 connection(s)
I20260926 06:21:14.270341 14525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
I20260926 06:21:14.275265 14243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:14.270992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:35457 (local address 127.4.202.190:44003)
0926 06:21:14.271038 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:14.271040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:14.271225 (+ 185us) server_negotiation.cc:407] Connection header received
0926 06:21:14.271707 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:14.271708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:14.271720 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:14.271777 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:14.272460 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:14.272963 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:14.273908 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:14.274120 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:14.274208 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:14.274980 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:21:14.274981 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:21:14.274982 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:21:14.274986 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:21:14.275001 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:21:14.275004 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:21:14.275058 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 06:21:14.275161 (+ 103us) server_negotiation.cc:299] Negotiation successful
0926 06:21:14.275190 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 06:21:14.275490 14655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:14.270951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:44003 (local address 127.4.202.131:35457)
0926 06:21:14.271110 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:14.271146 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:14.271587 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:14.271836 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:14.271854 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:14.271864 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:14.272366 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:14.272370 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:14.273054 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:14.273055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:14.273804 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:14.273806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:14.274218 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:14.274219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:14.274347 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:14.274809 (+ 462us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:21:14.274809 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:14.274833 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:21:14.274867 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:21:14.275061 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:21:14.275066 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:21:14.275069 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:21:14.275149 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 06:21:14.275262 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":66,"threads_started":1}
I20260926 06:21:14.275979 14653 heartbeater.cc:347] Connected to a master server at 127.4.202.190:44003
I20260926 06:21:14.276052 14653 heartbeater.cc:464] Registering TS with master...
I20260926 06:21:14.276233 14653 heartbeater.cc:511] Master 127.4.202.190:44003 requested a full tablet report, sending...
I20260926 06:21:14.276592 14203 ts_manager.cc:194] Registered new tserver with Master: c99637c2286a4b1d9cf9bcbdc9d7589c (127.4.202.131:46359)
I20260926 06:21:14.276963 14203 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.202.131:35457
I20260926 06:21:14.277700 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 14525
I20260926 06:21:14.277771 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance
I20260926 06:21:14.277910 4906 external_mini_cluster.cc:1428] Setting key 55e70cff8ea11b9889f64032329533b7
I20260926 06:21:14.278849 4906 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 06:21:14.280550 14243 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:14.280101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40710 (local address 127.4.202.190:44003)
0926 06:21:14.280169 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:14.280170 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:14.280226 (+ 56us) server_negotiation.cc:407] Connection header received
0926 06:21:14.280263 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:14.280263 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:14.280273 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:14.280316 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:14.280501 (+ 185us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40710: BlockingRecv error: recv got EOF from 127.0.0.1:40710 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 06:21:14.280618 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 14263
I20260926 06:21:14.285326 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 14394
I20260926 06:21:14.289170 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 14525
I20260926 06:21:14.293033 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 14172
2026-09-26T06:21:14Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (572 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-26T06:21:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:21:14Z Disabled control of system clock
I20260926 06:21:14.318892 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:42601
--webserver_interface=127.4.202.190
--webserver_port=0
--builtin_ntp_servers=127.4.202.148:42197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:42601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 06:21:14.398497 14664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:21:14.398715 14664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:21:14.398736 14664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:21:14.399811 14664 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:21:14.399842 14664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:21:14.399855 14664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:21:14.399868 14664 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:21:14.399879 14664 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:21:14.401124 14664 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:42197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:42601
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:42601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:21:14.401300 14664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:21:14.401523 14664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:21:14.406250 14670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:21:14.406212 14672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:21:14.406212 14669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:21:14.406400 14664 server_base.cc:1034] running on GCE node
I20260926 06:21:14.406795 14664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:21:14.407109 14664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:21:14.408283 14664 hybrid_clock.cc:648] HybridClock initialized: now 1790403674408272 us; error 27 us; skew 500 ppm
I20260926 06:21:14.409493 14664 webserver.cc:460] Webserver started at http://127.4.202.190:33999/ using document root <none> and password file <none>
I20260926 06:21:14.409735 14664 fs_manager.cc:359] Metadata directory not provided
I20260926 06:21:14.409802 14664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:21:14.409943 14664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:21:14.411038 14664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "b83d736ae5674efdaa7e75ec7207e776"
format_stamp: "Formatted at 2026-09-26 06:21:14 on dist-test-slave-pd0t"
server_key: "9cb70dcdafc9dcdfe9a1bd8b6319ee26"
server_key_iv: "151f3959b9c1d1f30ede7b2d3447de05"
server_key_version: "encryptionkey@0"
I20260926 06:21:14.411384 14664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "b83d736ae5674efdaa7e75ec7207e776"
format_stamp: "Formatted at 2026-09-26 06:21:14 on dist-test-slave-pd0t"
server_key: "9cb70dcdafc9dcdfe9a1bd8b6319ee26"
server_key_iv: "151f3959b9c1d1f30ede7b2d3447de05"
server_key_version: "encryptionkey@0"
I20260926 06:21:14.412679 14664 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 06:21:14.413415 14678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:14.413594 14664 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:21:14.413684 14664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "b83d736ae5674efdaa7e75ec7207e776"
format_stamp: "Formatted at 2026-09-26 06:21:14 on dist-test-slave-pd0t"
server_key: "9cb70dcdafc9dcdfe9a1bd8b6319ee26"
server_key_iv: "151f3959b9c1d1f30ede7b2d3447de05"
server_key_version: "encryptionkey@0"
I20260926 06:21:14.413735 14664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:21:14.416941 14664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:21:14.417132 14664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:21:14.417232 14664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:21:14.420814 14664 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:42601
I20260926 06:21:14.420881 14730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:42601 every 8 connection(s)
I20260926 06:21:14.421175 14664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:21:14.421864 14731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:21:14.422693 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 14664
I20260926 06:21:14.422756 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:21:14.422925 4906 external_mini_cluster.cc:1428] Setting key b69d27e785e3f6f5c38b97a14933c40c
I20260926 06:21:14.425364 14731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b83d736ae5674efdaa7e75ec7207e776: Bootstrap starting.
I20260926 06:21:14.425415 14734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:14.423390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43438 (local address 127.4.202.190:42601)
0926 06:21:14.423589 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:14.423597 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:14.423632 (+ 35us) server_negotiation.cc:407] Connection header received
0926 06:21:14.424165 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:14.424173 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:14.424234 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:14.424390 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:14.424985 (+ 595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:21:14.424989 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:21:14.424989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:21:14.424993 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:21:14.425043 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:21:14.425055 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:21:14.425127 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 06:21:14.425184 (+ 57us) server_negotiation.cc:299] Negotiation successful
0926 06:21:14.425235 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":83,"threads_started":1}
I20260926 06:21:14.426033 14731 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b83d736ae5674efdaa7e75ec7207e776: Neither blocks nor log segments found. Creating new log.
I20260926 06:21:14.426275 14731 log.cc:824] T 00000000000000000000000000000000 P b83d736ae5674efdaa7e75ec7207e776: Log is configured to *not* fsync() on all Append() calls
I20260926 06:21:14.426959 14731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b83d736ae5674efdaa7e75ec7207e776: No bootstrap required, opened a new log
I20260926 06:21:14.428159 14731 raft_consensus.cc:348] T 00000000000000000000000000000000 P b83d736ae5674efdaa7e75ec7207e776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b83d736ae5674efdaa7e75ec7207e776" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42601 } }
I20260926 06:21:14.428298 14731 raft_consensus.cc:374] T 00000000000000000000000000000000 P b83d736ae5674efdaa7e75ec7207e776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:21:14.428320 14731 raft_consensus.cc:729] T 00000000000000000000000000000000 P b83d736ae5674efdaa7e75ec7207e776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b83d736ae5674efdaa7e75ec7207e776, State: Initialized, Role: FOLLOWER
I20260926 06:21:14.428434 14731 consensus_queue.cc:260] T 00000000000000000000000000000000 P b83d736ae5674efdaa7e75ec7207e776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b83d736ae5674efdaa7e75ec7207e776" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42601 } }
I20260926 06:21:14.428512 14731 raft_consensus.cc:388] T 00000000000000000000000000000000 P b83d736ae5674efdaa7e75ec7207e776 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:21:14.428535 14731 raft_consensus.cc:482] T 00000000000000000000000000000000 P b83d736ae5674efdaa7e75ec7207e776 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:21:14.428561 14731 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b83d736ae5674efdaa7e75ec7207e776 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:21:14.429280 14731 raft_consensus.cc:504] T 00000000000000000000000000000000 P b83d736ae5674efdaa7e75ec7207e776 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b83d736ae5674efdaa7e75ec7207e776" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42601 } }
I20260926 06:21:14.429364 14731 leader_election.cc:302] T 00000000000000000000000000000000 P b83d736ae5674efdaa7e75ec7207e776 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b83d736ae5674efdaa7e75ec7207e776; no voters:
I20260926 06:21:14.429476 14731 leader_election.cc:288] T 00000000000000000000000000000000 P b83d736ae5674efdaa7e75ec7207e776 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:21:14.429548 14736 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b83d736ae5674efdaa7e75ec7207e776 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:21:14.429713 14736 raft_consensus.cc:686] T 00000000000000000000000000000000 P b83d736ae5674efdaa7e75ec7207e776 [term 1 LEADER]: Becoming Leader. State: Replica: b83d736ae5674efdaa7e75ec7207e776, State: Running, Role: LEADER
I20260926 06:21:14.429729 14731 sys_catalog.cc:564] T 00000000000000000000000000000000 P b83d736ae5674efdaa7e75ec7207e776 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:21:14.429807 14736 consensus_queue.cc:237] T 00000000000000000000000000000000 P b83d736ae5674efdaa7e75ec7207e776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b83d736ae5674efdaa7e75ec7207e776" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42601 } }
I20260926 06:21:14.430281 14737 sys_catalog.cc:455] T 00000000000000000000000000000000 P b83d736ae5674efdaa7e75ec7207e776 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b83d736ae5674efdaa7e75ec7207e776" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b83d736ae5674efdaa7e75ec7207e776" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42601 } } }
I20260926 06:21:14.430281 14738 sys_catalog.cc:455] T 00000000000000000000000000000000 P b83d736ae5674efdaa7e75ec7207e776 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b83d736ae5674efdaa7e75ec7207e776. Latest consensus state: current_term: 1 leader_uuid: "b83d736ae5674efdaa7e75ec7207e776" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b83d736ae5674efdaa7e75ec7207e776" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42601 } } }
I20260926 06:21:14.430435 14737 sys_catalog.cc:458] T 00000000000000000000000000000000 P b83d736ae5674efdaa7e75ec7207e776 [sys.catalog]: This master's current role is: LEADER
I20260926 06:21:14.430450 14738 sys_catalog.cc:458] T 00000000000000000000000000000000 P b83d736ae5674efdaa7e75ec7207e776 [sys.catalog]: This master's current role is: LEADER
I20260926 06:21:14.430745 14745 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:21:14.431190 14745 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:21:14.435561 14745 catalog_manager.cc:1347] Generated new cluster ID: 53c56381361c4d1a96f6177e0d308a73
I20260926 06:21:14.435604 14745 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:21:14.445556 14745 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:21:14.445701 14745 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:21:14.454700 14745 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b83d736ae5674efdaa7e75ec7207e776: Generated new TSK 0
I20260926 06:21:14.454878 14745 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:21:14.458786 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.129:0
--local_ip_for_outbound_sockets=127.4.202.129
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:42601
--builtin_ntp_servers=127.4.202.148:42197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 06:21:14.563311 14755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:21:14.563524 14755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:21:14.563560 14755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:21:14.564729 14755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:21:14.564790 14755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:21:14.564832 14755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.129
I20260926 06:21:14.566141 14755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:42197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:42601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.129
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:21:14.566388 14755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:21:14.566715 14755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:21:14.571437 14760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:21:14.571558 14761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:21:14.571465 14763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:21:14.571942 14755 server_base.cc:1034] running on GCE node
I20260926 06:21:14.572124 14755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:21:14.572368 14755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:21:14.573566 14755 hybrid_clock.cc:648] HybridClock initialized: now 1790403674573552 us; error 26 us; skew 500 ppm
I20260926 06:21:14.580813 14755 webserver.cc:460] Webserver started at http://127.4.202.129:46313/ using document root <none> and password file <none>
I20260926 06:21:14.581054 14755 fs_manager.cc:359] Metadata directory not provided
I20260926 06:21:14.581111 14755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:21:14.581233 14755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:21:14.582512 14755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/data/instance:
uuid: "5d79f4955ac740c8992cc5709bf10b33"
format_stamp: "Formatted at 2026-09-26 06:21:14 on dist-test-slave-pd0t"
server_key: "b8d6578fd3df2138bbfb8ca7f423e9fd"
server_key_iv: "0594c2b85e22580d3e2c42e32c558bc7"
server_key_version: "encryptionkey@0"
I20260926 06:21:14.582952 14755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance:
uuid: "5d79f4955ac740c8992cc5709bf10b33"
format_stamp: "Formatted at 2026-09-26 06:21:14 on dist-test-slave-pd0t"
server_key: "b8d6578fd3df2138bbfb8ca7f423e9fd"
server_key_iv: "0594c2b85e22580d3e2c42e32c558bc7"
server_key_version: "encryptionkey@0"
I20260926 06:21:14.584416 14755 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:21:14.585038 14769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:14.585199 14755 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:21:14.585276 14755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/wal
uuid: "5d79f4955ac740c8992cc5709bf10b33"
format_stamp: "Formatted at 2026-09-26 06:21:14 on dist-test-slave-pd0t"
server_key: "b8d6578fd3df2138bbfb8ca7f423e9fd"
server_key_iv: "0594c2b85e22580d3e2c42e32c558bc7"
server_key_version: "encryptionkey@0"
I20260926 06:21:14.585336 14755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:21:14.588416 14755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:21:14.588629 14755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:21:14.588719 14755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:21:14.588874 14755 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:21:14.589097 14755 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:21:14.589128 14755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:14.589145 14755 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:21:14.589157 14755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:14.594434 14755 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.129:34391
I20260926 06:21:14.594506 14882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.129:34391 every 8 connection(s)
I20260926 06:21:14.594821 14755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
I20260926 06:21:14.597625 14734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:14.595494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:40465 (local address 127.4.202.190:42601)
0926 06:21:14.595536 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:14.595538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:14.595772 (+ 234us) server_negotiation.cc:407] Connection header received
0926 06:21:14.596226 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:14.596227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:14.596234 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:14.596289 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:14.597336 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:21:14.597337 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:21:14.597338 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:21:14.597342 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:21:14.597356 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:21:14.597360 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:21:14.597455 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 06:21:14.597546 (+ 91us) server_negotiation.cc:299] Negotiation successful
0926 06:21:14.597561 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:21:14.597808 14885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:14.595486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42601 (local address 127.4.202.129:40465)
0926 06:21:14.595662 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:14.595691 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:14.596126 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:14.596351 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:14.596355 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:14.596369 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:14.597196 (+ 827us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:21:14.597196 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:14.597222 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:21:14.597247 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:21:14.597453 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:21:14.597459 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:21:14.597462 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:21:14.597556 (+ 94us) client_negotiation.cc:240] Negotiation successful
0926 06:21:14.597612 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":83,"threads_started":1}
I20260926 06:21:14.598304 14883 heartbeater.cc:347] Connected to a master server at 127.4.202.190:42601
I20260926 06:21:14.598371 14883 heartbeater.cc:464] Registering TS with master...
I20260926 06:21:14.598517 14883 heartbeater.cc:511] Master 127.4.202.190:42601 requested a full tablet report, sending...
I20260926 06:21:14.599004 14695 ts_manager.cc:194] Registered new tserver with Master: 5d79f4955ac740c8992cc5709bf10b33 (127.4.202.129:34391)
I20260926 06:21:14.604001 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 14755
I20260926 06:21:14.604068 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance
I20260926 06:21:14.604245 4906 external_mini_cluster.cc:1428] Setting key 92fc7da5f9f50b1291d1a68dde09c3d7
I20260926 06:21:14.605083 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.130:0
--local_ip_for_outbound_sockets=127.4.202.130
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:42601
--builtin_ntp_servers=127.4.202.148:42197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 06:21:14.682911 14886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:21:14.683143 14886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:21:14.683171 14886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:21:14.684249 14886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:21:14.684283 14886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:21:14.684307 14886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.130
I20260926 06:21:14.685556 14886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:42197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:42601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.130
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:21:14.685709 14886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:21:14.685905 14886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:21:14.691032 14894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:21:14.691102 14891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:21:14.691134 14892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:21:14.691370 14886 server_base.cc:1034] running on GCE node
I20260926 06:21:14.691524 14886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:21:14.691695 14886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:21:14.692830 14886 hybrid_clock.cc:648] HybridClock initialized: now 1790403674692818 us; error 26 us; skew 500 ppm
I20260926 06:21:14.694113 14886 webserver.cc:460] Webserver started at http://127.4.202.130:41057/ using document root <none> and password file <none>
I20260926 06:21:14.694285 14886 fs_manager.cc:359] Metadata directory not provided
I20260926 06:21:14.694324 14886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:21:14.694429 14886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:21:14.695429 14886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/data/instance:
uuid: "4de8e03e7c254e2db3b945fb27f91a07"
format_stamp: "Formatted at 2026-09-26 06:21:14 on dist-test-slave-pd0t"
server_key: "5bf4e01d851d2121061d524541c47521"
server_key_iv: "0e7016cad0a7dbf8917e1efca0ed7d7e"
server_key_version: "encryptionkey@0"
I20260926 06:21:14.695709 14886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance:
uuid: "4de8e03e7c254e2db3b945fb27f91a07"
format_stamp: "Formatted at 2026-09-26 06:21:14 on dist-test-slave-pd0t"
server_key: "5bf4e01d851d2121061d524541c47521"
server_key_iv: "0e7016cad0a7dbf8917e1efca0ed7d7e"
server_key_version: "encryptionkey@0"
I20260926 06:21:14.696938 14886 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 06:21:14.697598 14900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:14.697749 14886 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:21:14.697835 14886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/wal
uuid: "4de8e03e7c254e2db3b945fb27f91a07"
format_stamp: "Formatted at 2026-09-26 06:21:14 on dist-test-slave-pd0t"
server_key: "5bf4e01d851d2121061d524541c47521"
server_key_iv: "0e7016cad0a7dbf8917e1efca0ed7d7e"
server_key_version: "encryptionkey@0"
I20260926 06:21:14.697888 14886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:21:14.701009 14886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:21:14.701238 14886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:21:14.701334 14886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:21:14.701479 14886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:21:14.701704 14886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:21:14.701735 14886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:14.701752 14886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:21:14.701764 14886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:14.707119 14886 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.130:45169
I20260926 06:21:14.707185 15013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.130:45169 every 8 connection(s)
I20260926 06:21:14.707468 14886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
I20260926 06:21:14.708978 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 14886
I20260926 06:21:14.709080 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance
I20260926 06:21:14.709272 4906 external_mini_cluster.cc:1428] Setting key 71deca37af370b0b2c37786f6bee5f0b
I20260926 06:21:14.710167 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.131:0
--local_ip_for_outbound_sockets=127.4.202.131
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:42601
--builtin_ntp_servers=127.4.202.148:42197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260926 06:21:14.710557 14734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:14.708156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:47621 (local address 127.4.202.190:42601)
0926 06:21:14.708216 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:14.708218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:14.708447 (+ 229us) server_negotiation.cc:407] Connection header received
0926 06:21:14.708877 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:14.708878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:14.708883 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:14.708943 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:14.710256 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:21:14.710257 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:21:14.710257 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:21:14.710262 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:21:14.710277 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:21:14.710280 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:21:14.710363 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 06:21:14.710473 (+ 110us) server_negotiation.cc:299] Negotiation successful
0926 06:21:14.710487 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 06:21:14.710829 15016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:14.708090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42601 (local address 127.4.202.130:47621)
0926 06:21:14.708274 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:14.708310 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:14.708750 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:14.709015 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:14.709020 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:14.709030 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:14.710091 (+ 1061us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:21:14.710091 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:14.710117 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:21:14.710145 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:21:14.710356 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:21:14.710362 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:21:14.710365 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:21:14.710484 (+ 119us) client_negotiation.cc:240] Negotiation successful
0926 06:21:14.710544 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":79,"threads_started":1}
I20260926 06:21:14.711320 15014 heartbeater.cc:347] Connected to a master server at 127.4.202.190:42601
I20260926 06:21:14.711418 15014 heartbeater.cc:464] Registering TS with master...
I20260926 06:21:14.711567 15014 heartbeater.cc:511] Master 127.4.202.190:42601 requested a full tablet report, sending...
I20260926 06:21:14.711989 14695 ts_manager.cc:194] Registered new tserver with Master: 4de8e03e7c254e2db3b945fb27f91a07 (127.4.202.130:45169)
W20260926 06:21:14.788275 15017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:21:14.788460 15017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:21:14.788481 15017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:21:14.789628 15017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:21:14.789665 15017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:21:14.789690 15017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.131
I20260926 06:21:14.791000 15017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:42197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:42601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.131
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:21:14.791173 15017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:21:14.791389 15017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:21:14.796123 15022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:21:14.796137 15023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:21:14.796234 15025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:21:14.796262 15017 server_base.cc:1034] running on GCE node
I20260926 06:21:14.796554 15017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:21:14.796808 15017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:21:14.797988 15017 hybrid_clock.cc:648] HybridClock initialized: now 1790403674797976 us; error 43 us; skew 500 ppm
I20260926 06:21:14.799311 15017 webserver.cc:460] Webserver started at http://127.4.202.131:35623/ using document root <none> and password file <none>
I20260926 06:21:14.799574 15017 fs_manager.cc:359] Metadata directory not provided
I20260926 06:21:14.799643 15017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:21:14.799774 15017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:21:14.800846 15017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/data/instance:
uuid: "4ad5f121268a414897ffbf71222ba614"
format_stamp: "Formatted at 2026-09-26 06:21:14 on dist-test-slave-pd0t"
server_key: "d440919e70037cdf2403b4b34e44e067"
server_key_iv: "60c860e0c258007d5b2cef382aa01b51"
server_key_version: "encryptionkey@0"
I20260926 06:21:14.801177 15017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance:
uuid: "4ad5f121268a414897ffbf71222ba614"
format_stamp: "Formatted at 2026-09-26 06:21:14 on dist-test-slave-pd0t"
server_key: "d440919e70037cdf2403b4b34e44e067"
server_key_iv: "60c860e0c258007d5b2cef382aa01b51"
server_key_version: "encryptionkey@0"
I20260926 06:21:14.802395 15017 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:21:14.803184 15031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:14.803370 15017 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:14.803458 15017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/wal
uuid: "4ad5f121268a414897ffbf71222ba614"
format_stamp: "Formatted at 2026-09-26 06:21:14 on dist-test-slave-pd0t"
server_key: "d440919e70037cdf2403b4b34e44e067"
server_key_iv: "60c860e0c258007d5b2cef382aa01b51"
server_key_version: "encryptionkey@0"
I20260926 06:21:14.803556 15017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:21:14.806660 15017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:21:14.806910 15017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:21:14.807032 15017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:21:14.807229 15017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:21:14.807502 15017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:21:14.807531 15017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:14.807581 15017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:21:14.807607 15017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:14.812973 15017 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.131:40001
I20260926 06:21:14.813040 15144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.131:40001 every 8 connection(s)
I20260926 06:21:14.813321 15017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
I20260926 06:21:14.814792 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 15017
I20260926 06:21:14.814896 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance
I20260926 06:21:14.815071 4906 external_mini_cluster.cc:1428] Setting key fe6abbb45a2956f50e299e99646eca4d
I20260926 06:21:14.816612 14734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:14.814101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:56011 (local address 127.4.202.190:42601)
0926 06:21:14.814143 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:14.814145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:14.814361 (+ 216us) server_negotiation.cc:407] Connection header received
0926 06:21:14.814945 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:14.814946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:14.814952 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:14.815009 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:14.816350 (+ 1341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:21:14.816351 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:21:14.816352 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:21:14.816356 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:21:14.816369 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:21:14.816372 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:21:14.816449 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 06:21:14.816533 (+ 84us) server_negotiation.cc:299] Negotiation successful
0926 06:21:14.816546 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 06:21:14.816797 15147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:14.814065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42601 (local address 127.4.202.131:56011)
0926 06:21:14.814237 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:14.814273 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:14.814790 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:14.815077 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:14.815081 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:14.815092 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:14.816203 (+ 1111us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:21:14.816204 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:14.816230 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:21:14.816255 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:21:14.816451 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:21:14.816455 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:21:14.816458 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:21:14.816541 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 06:21:14.816597 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":90,"threads_started":1}
I20260926 06:21:14.817230 15145 heartbeater.cc:347] Connected to a master server at 127.4.202.190:42601
I20260926 06:21:14.817298 15145 heartbeater.cc:464] Registering TS with master...
I20260926 06:21:14.817437 15145 heartbeater.cc:511] Master 127.4.202.190:42601 requested a full tablet report, sending...
I20260926 06:21:14.817777 14695 ts_manager.cc:194] Registered new tserver with Master: 4ad5f121268a414897ffbf71222ba614 (127.4.202.131:40001)
I20260926 06:21:14.826774 4906 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 06:21:14.828579 14734 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:21:14.828089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43448 (local address 127.4.202.190:42601)
0926 06:21:14.828140 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:14.828142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:14.828274 (+ 132us) server_negotiation.cc:407] Connection header received
0926 06:21:14.828310 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:14.828310 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:14.828314 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:14.828358 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:14.828521 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43448: BlockingRecv error: recv got EOF from 127.0.0.1:43448 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260926 06:21:14.828655 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 14755
I20260926 06:21:14.833580 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 14886
I20260926 06:21:14.837651 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 15017
I20260926 06:21:14.841629 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 14664
2026-09-26T06:21:14Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (548 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:21:14 dist-test-slave-pd0t krb5kdc[15155](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:21:14 dist-test-slave-pd0t krb5kdc[15155](info): set up 2 sockets
Sep 26 06:21:14 dist-test-slave-pd0t krb5kdc[15155](info): commencing operation
krb5kdc: starting...
W20260926 06:21:15.866837 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:21:15 dist-test-slave-pd0t krb5kdc[15155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:21:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:21:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.202.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
I20260926 06:21:15.978173 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:44583
--webserver_interface=127.4.202.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:34785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:44583
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:21:16.055912 15168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:21:16.056099 15168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:21:16.056121 15168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:21:16.057168 15168 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:21:16.057199 15168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:21:16.057212 15168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:21:16.057224 15168 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:21:16.057235 15168 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:21:16.058622 15168 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:34785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:44583
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:44583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:21:16.058794 15168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:21:16.058943 15168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 06:21:16.063578 15168 server_base.cc:1034] running on GCE node
W20260926 06:21:16.063618 15176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:21:16.063719 15174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:21:16.063632 15173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:21:16.064029 15168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:21:16.064311 15168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:21:16.065467 15168 hybrid_clock.cc:648] HybridClock initialized: now 1790403676065446 us; error 39 us; skew 500 ppm
Sep 26 06:21:16 dist-test-slave-pd0t krb5kdc[15155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403676, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:21:16.067795 15168 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:21:16.068217 15168 webserver.cc:460] Webserver started at http://127.4.202.190:34879/ using document root <none> and password file <none>
I20260926 06:21:16.068451 15168 fs_manager.cc:359] Metadata directory not provided
I20260926 06:21:16.068516 15168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:21:16.068655 15168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:21:16.069610 15168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "0e0afceed4334e75841df71fabcb7787"
format_stamp: "Formatted at 2026-09-26 06:21:16 on dist-test-slave-pd0t"
server_key: "d45ffb49e42bcd9817e4c33e5b5295cd"
server_key_iv: "197cbfec95e3656ee586907973cc6368"
server_key_version: "encryptionkey@0"
I20260926 06:21:16.069962 15168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "0e0afceed4334e75841df71fabcb7787"
format_stamp: "Formatted at 2026-09-26 06:21:16 on dist-test-slave-pd0t"
server_key: "d45ffb49e42bcd9817e4c33e5b5295cd"
server_key_iv: "197cbfec95e3656ee586907973cc6368"
server_key_version: "encryptionkey@0"
I20260926 06:21:16.071290 15168 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260926 06:21:16.072024 15183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:16.072186 15168 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:21:16.072279 15168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "0e0afceed4334e75841df71fabcb7787"
format_stamp: "Formatted at 2026-09-26 06:21:16 on dist-test-slave-pd0t"
server_key: "d45ffb49e42bcd9817e4c33e5b5295cd"
server_key_iv: "197cbfec95e3656ee586907973cc6368"
server_key_version: "encryptionkey@0"
I20260926 06:21:16.072355 15168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:21:16.105139 15168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:21:16.106980 15168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:21:16.107126 15168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:21:16.110885 15168 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:44583
I20260926 06:21:16.110916 15235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:44583 every 8 connection(s)
I20260926 06:21:16.111260 15168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:21:16.112178 15236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:21:16.112421 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 15168
I20260926 06:21:16.112522 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:21:16.112697 4906 external_mini_cluster.cc:1428] Setting key fe75d163ce01e7b23dcee9147178bfe7
I20260926 06:21:16.115516 15236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e0afceed4334e75841df71fabcb7787: Bootstrap starting.
I20260926 06:21:16.116230 15236 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e0afceed4334e75841df71fabcb7787: Neither blocks nor log segments found. Creating new log.
I20260926 06:21:16.116526 15236 log.cc:824] T 00000000000000000000000000000000 P 0e0afceed4334e75841df71fabcb7787: Log is configured to *not* fsync() on all Append() calls
I20260926 06:21:16.117184 15236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e0afceed4334e75841df71fabcb7787: No bootstrap required, opened a new log
Sep 26 06:21:16 dist-test-slave-pd0t krb5kdc[15155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:21:16.119365 15236 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e0afceed4334e75841df71fabcb7787 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e0afceed4334e75841df71fabcb7787" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 44583 } }
I20260926 06:21:16.119530 15236 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e0afceed4334e75841df71fabcb7787 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:21:16.119556 15236 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e0afceed4334e75841df71fabcb7787 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e0afceed4334e75841df71fabcb7787, State: Initialized, Role: FOLLOWER
I20260926 06:21:16.119699 15236 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e0afceed4334e75841df71fabcb7787 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e0afceed4334e75841df71fabcb7787" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 44583 } }
I20260926 06:21:16.119776 15236 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e0afceed4334e75841df71fabcb7787 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:21:16.119797 15236 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e0afceed4334e75841df71fabcb7787 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:21:16.119874 15236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e0afceed4334e75841df71fabcb7787 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:21:16.120582 15236 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e0afceed4334e75841df71fabcb7787 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e0afceed4334e75841df71fabcb7787" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 44583 } }
I20260926 06:21:16.120693 15236 leader_election.cc:302] T 00000000000000000000000000000000 P 0e0afceed4334e75841df71fabcb7787 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e0afceed4334e75841df71fabcb7787; no voters:
I20260926 06:21:16.120844 15236 leader_election.cc:288] T 00000000000000000000000000000000 P 0e0afceed4334e75841df71fabcb7787 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:21:16.120914 15241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e0afceed4334e75841df71fabcb7787 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:21:16.121078 15241 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e0afceed4334e75841df71fabcb7787 [term 1 LEADER]: Becoming Leader. State: Replica: 0e0afceed4334e75841df71fabcb7787, State: Running, Role: LEADER
I20260926 06:21:16.121125 15236 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e0afceed4334e75841df71fabcb7787 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:21:16.121232 15241 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e0afceed4334e75841df71fabcb7787 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e0afceed4334e75841df71fabcb7787" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 44583 } }
I20260926 06:21:16.121769 15243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e0afceed4334e75841df71fabcb7787 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e0afceed4334e75841df71fabcb7787. Latest consensus state: current_term: 1 leader_uuid: "0e0afceed4334e75841df71fabcb7787" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e0afceed4334e75841df71fabcb7787" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 44583 } } }
I20260926 06:21:16.121873 15243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e0afceed4334e75841df71fabcb7787 [sys.catalog]: This master's current role is: LEADER
I20260926 06:21:16.122280 15249 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:21:16.122259 15242 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e0afceed4334e75841df71fabcb7787 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e0afceed4334e75841df71fabcb7787" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e0afceed4334e75841df71fabcb7787" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 44583 } } }
I20260926 06:21:16.122654 15242 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e0afceed4334e75841df71fabcb7787 [sys.catalog]: This master's current role is: LEADER
I20260926 06:21:16.122973 15249 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:21:16.123734 15239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:16.113203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37488 (local address 127.4.202.190:44583)
0926 06:21:16.113372 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:16.113380 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:16.113412 (+ 32us) server_negotiation.cc:407] Connection header received
0926 06:21:16.113818 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:16.113824 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:16.113865 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:16.113980 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:16.114785 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.115670 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.116731 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.117235 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.117331 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:16.120005 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:21:16.120009 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:21:16.120010 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:21:16.120021 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:21:16.121960 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:21:16.122353 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:21:16.122355 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:21:16.122356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:21:16.122449 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:21:16.122675 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:21:16.122676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:21:16.122676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:21:16.122912 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:21:16.123048 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0926 06:21:16.123429 (+ 381us) server_negotiation.cc:299] Negotiation successful
0926 06:21:16.123552 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":89,"threads_started":1}
I20260926 06:21:16.124611 15249 catalog_manager.cc:1347] Generated new cluster ID: e895791bce4444d78a666b814bb9b98d
I20260926 06:21:16.124658 15249 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:21:16.142238 15249 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:21:16.142937 15249 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:21:16.154922 15249 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e0afceed4334e75841df71fabcb7787: Generated new TSK 0
I20260926 06:21:16.155085 15249 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.202.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
I20260926 06:21:16.182019 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.129:0
--local_ip_for_outbound_sockets=127.4.202.129
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:44583
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:34785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:21:16.262573 15262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:21:16.262805 15262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:21:16.262832 15262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:21:16.263953 15262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:21:16.263991 15262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:21:16.264016 15262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.129
I20260926 06:21:16.265269 15262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:34785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:44583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.129
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:21:16.265437 15262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:21:16.265641 15262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:21:16.270249 15268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:21:16.270381 15270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:21:16.270483 15267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:21:16.270427 15262 server_base.cc:1034] running on GCE node
I20260926 06:21:16.270744 15262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:21:16.270943 15262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:21:16.272104 15262 hybrid_clock.cc:648] HybridClock initialized: now 1790403676272081 us; error 31 us; skew 500 ppm
Sep 26 06:21:16 dist-test-slave-pd0t krb5kdc[15155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403676, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:21:16.274616 15262 init.cc:377] Logged in from keytab as kudu/127.4.202.129@KRBTEST.COM (short username kudu)
I20260926 06:21:16.275059 15262 webserver.cc:460] Webserver started at http://127.4.202.129:42683/ using document root <none> and password file <none>
I20260926 06:21:16.275261 15262 fs_manager.cc:359] Metadata directory not provided
I20260926 06:21:16.275301 15262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:21:16.275393 15262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:21:16.276412 15262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/data/instance:
uuid: "45852b454b4644a4becf4fef10318127"
format_stamp: "Formatted at 2026-09-26 06:21:16 on dist-test-slave-pd0t"
server_key: "3e19d31808b67720a28828a0bd054668"
server_key_iv: "f6d16fedfbe199c125872eab82a78efe"
server_key_version: "encryptionkey@0"
I20260926 06:21:16.276690 15262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance:
uuid: "45852b454b4644a4becf4fef10318127"
format_stamp: "Formatted at 2026-09-26 06:21:16 on dist-test-slave-pd0t"
server_key: "3e19d31808b67720a28828a0bd054668"
server_key_iv: "f6d16fedfbe199c125872eab82a78efe"
server_key_version: "encryptionkey@0"
I20260926 06:21:16.277894 15262 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 06:21:16.278519 15277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:16.278700 15262 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:16.278797 15262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/wal
uuid: "45852b454b4644a4becf4fef10318127"
format_stamp: "Formatted at 2026-09-26 06:21:16 on dist-test-slave-pd0t"
server_key: "3e19d31808b67720a28828a0bd054668"
server_key_iv: "f6d16fedfbe199c125872eab82a78efe"
server_key_version: "encryptionkey@0"
I20260926 06:21:16.278877 15262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:21:16.292400 15262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:21:16.294067 15262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:21:16.294219 15262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:21:16.294440 15262 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:21:16.294781 15262 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:21:16.294833 15262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:16.294867 15262 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:21:16.294895 15262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:16.300549 15262 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.129:39499
I20260926 06:21:16.300588 15390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.129:39499 every 8 connection(s)
I20260926 06:21:16.300868 15262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
I20260926 06:21:16.306243 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 15262
I20260926 06:21:16.306349 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance
Sep 26 06:21:16 dist-test-slave-pd0t krb5kdc[15155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403676, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:21:16.306543 4906 external_mini_cluster.cc:1428] Setting key 1433f932229c5d0a88a2028a972f6c42
I20260926 06:21:16.309671 15239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:16.301548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:42649 (local address 127.4.202.190:44583)
0926 06:21:16.301629 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:16.301631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:16.301779 (+ 148us) server_negotiation.cc:407] Connection header received
0926 06:21:16.302199 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:16.302200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:16.302210 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:16.302257 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:16.303181 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.303661 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.304604 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.304796 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.304847 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:16.306982 (+ 2135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:21:16.306983 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:21:16.306984 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:21:16.306991 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:21:16.308303 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:21:16.308703 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:21:16.308704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:21:16.308704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:21:16.308764 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:21:16.309023 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:21:16.309024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:21:16.309025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:21:16.309143 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:21:16.309214 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 06:21:16.309495 (+ 281us) server_negotiation.cc:299] Negotiation successful
0926 06:21:16.309583 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 06:21:16.309820 15393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:16.301494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:44583 (local address 127.4.202.129:42649)
0926 06:21:16.301660 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:16.301693 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:16.302119 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:16.302311 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:16.302314 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:16.302927 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:16.303103 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:16.303107 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.303738 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.303738 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:16.304526 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:16.304529 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.304868 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.304869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:16.304960 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:16.305386 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:16.305394 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:16.306872 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:21:16.308393 (+ 1521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:21:16.308400 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:21:16.308402 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:21:16.308618 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:21:16.308841 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:21:16.308841 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:21:16.308842 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:21:16.308952 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:21:16.309220 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:21:16.309226 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:21:16.309359 (+ 133us) client_negotiation.cc:769] Sending connection context
0926 06:21:16.309458 (+ 99us) client_negotiation.cc:240] Negotiation successful
0926 06:21:16.309604 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":107,"threads_started":1}
I20260926 06:21:16.310316 15391 heartbeater.cc:347] Connected to a master server at 127.4.202.190:44583
I20260926 06:21:16.310415 15391 heartbeater.cc:464] Registering TS with master...
I20260926 06:21:16.310643 15391 heartbeater.cc:511] Master 127.4.202.190:44583 requested a full tablet report, sending...
I20260926 06:21:16.311195 15200 ts_manager.cc:194] Registered new tserver with Master: 45852b454b4644a4becf4fef10318127 (127.4.202.129:39499)
I20260926 06:21:16.312161 15200 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.129@KRBTEST.COM'} at 127.4.202.129:42649
WARNING: no policy specified for kudu/127.4.202.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
I20260926 06:21:16.332950 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.130:0
--local_ip_for_outbound_sockets=127.4.202.130
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:44583
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:34785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:21:16.414917 15396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:21:16.415093 15396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:21:16.415115 15396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:21:16.416153 15396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:21:16.416186 15396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:21:16.416211 15396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.130
I20260926 06:21:16.417538 15396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:34785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:44583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.130
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:21:16.417713 15396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:21:16.417876 15396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:21:16.422750 15404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:21:16.422781 15402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:21:16.422868 15401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:21:16.423100 15396 server_base.cc:1034] running on GCE node
I20260926 06:21:16.423295 15396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:21:16.423521 15396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:21:16.424686 15396 hybrid_clock.cc:648] HybridClock initialized: now 1790403676424676 us; error 30 us; skew 500 ppm
Sep 26 06:21:16 dist-test-slave-pd0t krb5kdc[15155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403676, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:21:16.427134 15396 init.cc:377] Logged in from keytab as kudu/127.4.202.130@KRBTEST.COM (short username kudu)
I20260926 06:21:16.427584 15396 webserver.cc:460] Webserver started at http://127.4.202.130:34505/ using document root <none> and password file <none>
I20260926 06:21:16.427814 15396 fs_manager.cc:359] Metadata directory not provided
I20260926 06:21:16.427877 15396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:21:16.428021 15396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:21:16.428969 15396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/data/instance:
uuid: "f3d6dc2dbbb240fdbf0c66ce4158aee1"
format_stamp: "Formatted at 2026-09-26 06:21:16 on dist-test-slave-pd0t"
server_key: "8923e4c5676c286e393d61997e20155a"
server_key_iv: "d4412d03126a342201c73a23fa5f6251"
server_key_version: "encryptionkey@0"
I20260926 06:21:16.429307 15396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance:
uuid: "f3d6dc2dbbb240fdbf0c66ce4158aee1"
format_stamp: "Formatted at 2026-09-26 06:21:16 on dist-test-slave-pd0t"
server_key: "8923e4c5676c286e393d61997e20155a"
server_key_iv: "d4412d03126a342201c73a23fa5f6251"
server_key_version: "encryptionkey@0"
I20260926 06:21:16.430656 15396 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 06:21:16.431316 15411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:16.431481 15396 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:21:16.431573 15396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/wal
uuid: "f3d6dc2dbbb240fdbf0c66ce4158aee1"
format_stamp: "Formatted at 2026-09-26 06:21:16 on dist-test-slave-pd0t"
server_key: "8923e4c5676c286e393d61997e20155a"
server_key_iv: "d4412d03126a342201c73a23fa5f6251"
server_key_version: "encryptionkey@0"
I20260926 06:21:16.431655 15396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:21:16.443295 15396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:21:16.445356 15396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:21:16.445518 15396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:21:16.445711 15396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:21:16.445976 15396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:21:16.446027 15396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:16.446064 15396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:21:16.446090 15396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:16.452050 15396 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.130:37113
I20260926 06:21:16.452098 15524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.130:37113 every 8 connection(s)
I20260926 06:21:16.452373 15396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
I20260926 06:21:16.457080 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 15396
I20260926 06:21:16.457163 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance
I20260926 06:21:16.457342 4906 external_mini_cluster.cc:1428] Setting key a309ceef4d46024413174bb3540a3f70
Sep 26 06:21:16 dist-test-slave-pd0t krb5kdc[15155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403676, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:21:16.461503 15527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:16.453010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:44583 (local address 127.4.202.130:40053)
0926 06:21:16.453171 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:16.453209 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:16.453634 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:16.453844 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:16.453847 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:16.454382 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:16.454562 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:16.454567 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.455256 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.455256 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:16.456033 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:16.456035 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.456381 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.456382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:16.456478 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:16.456893 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:16.456900 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:16.458433 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:21:16.460214 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:21:16.460221 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:21:16.460223 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:21:16.460470 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:21:16.460675 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:21:16.460676 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:21:16.460676 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:21:16.460769 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:21:16.461056 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:21:16.461060 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:21:16.461162 (+ 102us) client_negotiation.cc:769] Sending connection context
0926 06:21:16.461241 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 06:21:16.461351 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":100,"threads_started":1}
I20260926 06:21:16.461519 15239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:16.453020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:40053 (local address 127.4.202.190:44583)
0926 06:21:16.453088 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:16.453090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:16.453290 (+ 200us) server_negotiation.cc:407] Connection header received
0926 06:21:16.453720 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:16.453721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:16.453730 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:16.453786 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:16.454694 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.455181 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.456114 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.456293 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.456347 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:16.458530 (+ 2183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:21:16.458531 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:21:16.458532 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:21:16.458539 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:21:16.460102 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:21:16.460546 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:21:16.460547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:21:16.460547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:21:16.460607 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:21:16.460850 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:21:16.460851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:21:16.460851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:21:16.460992 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:21:16.461061 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 06:21:16.461287 (+ 226us) server_negotiation.cc:299] Negotiation successful
0926 06:21:16.461377 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 06:21:16.461998 15525 heartbeater.cc:347] Connected to a master server at 127.4.202.190:44583
I20260926 06:21:16.462101 15525 heartbeater.cc:464] Registering TS with master...
I20260926 06:21:16.462260 15525 heartbeater.cc:511] Master 127.4.202.190:44583 requested a full tablet report, sending...
I20260926 06:21:16.462663 15200 ts_manager.cc:194] Registered new tserver with Master: f3d6dc2dbbb240fdbf0c66ce4158aee1 (127.4.202.130:37113)
I20260926 06:21:16.463191 15200 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.130@KRBTEST.COM'} at 127.4.202.130:40053
WARNING: no policy specified for kudu/127.4.202.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
I20260926 06:21:16.482015 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.131:0
--local_ip_for_outbound_sockets=127.4.202.131
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:44583
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:34785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:21:16.563053 15530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:21:16.563254 15530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:21:16.563292 15530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:21:16.564404 15530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:21:16.564464 15530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:21:16.564507 15530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.131
I20260926 06:21:16.565898 15530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:34785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:44583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.131
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:21:16.566157 15530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:21:16.566390 15530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:21:16.571110 15536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:21:16.571110 15535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:21:16.571175 15538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:21:16.571311 15530 server_base.cc:1034] running on GCE node
I20260926 06:21:16.571676 15530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:21:16.571918 15530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:21:16.573079 15530 hybrid_clock.cc:648] HybridClock initialized: now 1790403676573067 us; error 31 us; skew 500 ppm
Sep 26 06:21:16 dist-test-slave-pd0t krb5kdc[15155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403676, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:21:16.575809 15530 init.cc:377] Logged in from keytab as kudu/127.4.202.131@KRBTEST.COM (short username kudu)
I20260926 06:21:16.576267 15530 webserver.cc:460] Webserver started at http://127.4.202.131:37277/ using document root <none> and password file <none>
I20260926 06:21:16.576505 15530 fs_manager.cc:359] Metadata directory not provided
I20260926 06:21:16.576586 15530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:21:16.576713 15530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:21:16.577670 15530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/data/instance:
uuid: "518c1405cc8e4a4ab3dcd104cbee0051"
format_stamp: "Formatted at 2026-09-26 06:21:16 on dist-test-slave-pd0t"
server_key: "75e9f7bebbaaa5801a088ede56d039f8"
server_key_iv: "1c4c6788136b7bd1f0119fc297d15ce6"
server_key_version: "encryptionkey@0"
I20260926 06:21:16.578015 15530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance:
uuid: "518c1405cc8e4a4ab3dcd104cbee0051"
format_stamp: "Formatted at 2026-09-26 06:21:16 on dist-test-slave-pd0t"
server_key: "75e9f7bebbaaa5801a088ede56d039f8"
server_key_iv: "1c4c6788136b7bd1f0119fc297d15ce6"
server_key_version: "encryptionkey@0"
I20260926 06:21:16.579476 15530 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:21:16.580166 15545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:16.580325 15530 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:21:16.580417 15530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/wal
uuid: "518c1405cc8e4a4ab3dcd104cbee0051"
format_stamp: "Formatted at 2026-09-26 06:21:16 on dist-test-slave-pd0t"
server_key: "75e9f7bebbaaa5801a088ede56d039f8"
server_key_iv: "1c4c6788136b7bd1f0119fc297d15ce6"
server_key_version: "encryptionkey@0"
I20260926 06:21:16.580497 15530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:21:16.599855 15530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:21:16.602061 15530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:21:16.602216 15530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:21:16.602435 15530 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:21:16.602778 15530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:21:16.602833 15530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:16.602864 15530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:21:16.602893 15530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:16.608373 15530 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.131:40133
I20260926 06:21:16.608438 15658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.131:40133 every 8 connection(s)
I20260926 06:21:16.608757 15530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
Sep 26 06:21:16 dist-test-slave-pd0t krb5kdc[15155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403676, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:21:16.616503 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 15530
I20260926 06:21:16.616611 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance
I20260926 06:21:16.616796 4906 external_mini_cluster.cc:1428] Setting key 5fc3dd9491808faa3022a4f47cfa13d2
I20260926 06:21:16.617501 15239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:16.609479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:48305 (local address 127.4.202.190:44583)
0926 06:21:16.609523 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:16.609525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:16.609682 (+ 157us) server_negotiation.cc:407] Connection header received
0926 06:21:16.610117 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:16.610118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:16.610126 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:16.610180 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:16.611087 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.611584 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.612534 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.612720 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.612775 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:16.614860 (+ 2085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:21:16.614862 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:21:16.614862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:21:16.614869 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:21:16.616042 (+ 1173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:21:16.616418 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:21:16.616419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:21:16.616420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:21:16.616485 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:21:16.616788 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:21:16.616789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:21:16.616790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:21:16.616924 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:21:16.617013 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 06:21:16.617325 (+ 312us) server_negotiation.cc:299] Negotiation successful
0926 06:21:16.617409 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:21:16.617717 15661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:16.609419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:44583 (local address 127.4.202.131:48305)
0926 06:21:16.609563 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:16.609601 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:16.610043 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:16.610226 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:16.610229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:16.610813 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:16.611010 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:16.611014 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.611666 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.611666 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:16.612448 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:16.612450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.612795 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.612796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:16.612897 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:16.613476 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:16.613484 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:16.614770 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:21:16.616114 (+ 1344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:21:16.616119 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:21:16.616121 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:21:16.616332 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:21:16.616555 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:21:16.616556 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:21:16.616575 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:21:16.616727 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:21:16.617019 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:21:16.617026 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:21:16.617179 (+ 153us) client_negotiation.cc:769] Sending connection context
0926 06:21:16.617291 (+ 112us) client_negotiation.cc:240] Negotiation successful
0926 06:21:16.617478 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":81,"threads_started":1}
I20260926 06:21:16.618108 15659 heartbeater.cc:347] Connected to a master server at 127.4.202.190:44583
I20260926 06:21:16.618213 15659 heartbeater.cc:464] Registering TS with master...
I20260926 06:21:16.618381 15659 heartbeater.cc:511] Master 127.4.202.190:44583 requested a full tablet report, sending...
I20260926 06:21:16.618775 15200 ts_manager.cc:194] Registered new tserver with Master: 518c1405cc8e4a4ab3dcd104cbee0051 (127.4.202.131:40133)
I20260926 06:21:16.619170 15200 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.131@KRBTEST.COM'} at 127.4.202.131:48305
I20260926 06:21:16.628309 4906 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:21:16.635479 15239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:16.629546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37490 (local address 127.4.202.190:44583)
0926 06:21:16.629623 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:16.629625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:16.629713 (+ 88us) server_negotiation.cc:407] Connection header received
0926 06:21:16.629796 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:16.629797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:16.629807 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:16.629879 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:16.630434 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.630969 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.631692 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.631901 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.631984 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:16.633107 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:21:16.633108 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:21:16.633109 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:21:16.633115 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:21:16.634321 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:21:16.634695 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:21:16.634696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:21:16.634696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:21:16.634754 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:21:16.634951 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:21:16.634952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:21:16.634952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:21:16.635084 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:21:16.635192 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 06:21:16.635311 (+ 119us) server_negotiation.cc:299] Negotiation successful
0926 06:21:16.635384 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 06:21:16.636909 15200 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37490:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:21:16.638415 15200 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:21:16.644562 15667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:16.640599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:40133 (local address 127.0.0.1:46414)
0926 06:21:16.640949 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:16.640961 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:16.641017 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:16.641228 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:16.641229 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:16.641230 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:21:16.641266 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:16.641268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.642616 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.642616 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:16.643422 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:16.643424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.644281 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.644282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:16.644356 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:16.644368 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 06:21:16.644438 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 06:21:16.644490 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":4,"thread_start_us":45,"threads_started":1}
I20260926 06:21:16.644590 15670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:16.640736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46414 (local address 127.4.202.131:40133)
0926 06:21:16.641082 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:16.641086 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:16.641097 (+ 11us) server_negotiation.cc:407] Connection header received
0926 06:21:16.641133 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:16.641136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:16.641151 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:16.641232 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:21:16.641401 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.642480 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.643509 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.644181 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.644257 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:16.644306 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 06:21:16.644465 (+ 159us) server_negotiation.cc:299] Negotiation successful
0926 06:21:16.644512 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":71,"threads_started":1}
I20260926 06:21:16.644752 15671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:16.640746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59610 (local address 127.4.202.130:37113)
0926 06:21:16.640937 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:16.640943 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:16.640954 (+ 11us) server_negotiation.cc:407] Connection header received
0926 06:21:16.641022 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:16.641024 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:16.641039 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:16.641131 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:21:16.641280 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.642461 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.643750 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.644207 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.644300 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:16.644366 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 06:21:16.644605 (+ 239us) server_negotiation.cc:299] Negotiation successful
0926 06:21:16.644656 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":116,"threads_started":1}
I20260926 06:21:16.644873 15672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:16.640814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58568 (local address 127.4.202.129:39499)
0926 06:21:16.640997 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:16.641001 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:16.641193 (+ 192us) server_negotiation.cc:407] Connection header received
0926 06:21:16.641231 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:16.641234 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:16.641248 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:16.641313 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:21:16.641520 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.642237 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.643722 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.644450 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.644516 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:16.644567 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 06:21:16.644749 (+ 182us) server_negotiation.cc:299] Negotiation successful
0926 06:21:16.644791 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":78,"threads_started":1}
I20260926 06:21:16.644838 15668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:16.640616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:39499 (local address 127.0.0.1:58568)
0926 06:21:16.641088 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:16.641097 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:16.641139 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:16.641354 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:16.641355 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:16.641357 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:21:16.641430 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:16.641432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.642359 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.642361 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:16.643631 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:16.643634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.644603 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.644604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:16.644673 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:16.644677 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:21:16.644733 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 06:21:16.644776 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":54,"threads_started":1}
I20260926 06:21:16.645025 15669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:16.640599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:37113 (local address 127.0.0.1:59610)
0926 06:21:16.640826 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:16.640872 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:16.640939 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:16.641133 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:16.641135 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:16.641137 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:21:16.641195 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:16.641197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.642572 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.642587 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:16.643657 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:16.643658 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.644399 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.644400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:16.644460 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:16.644465 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 06:21:16.644511 (+ 46us) client_negotiation.cc:240] Negotiation successful
0926 06:21:16.644543 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":53,"threads_started":1}
I20260926 06:21:16.645824 15325 tablet_service.cc:1467] Processing CreateTablet for tablet b69214f6f73c48e1a814b494426fec52 (DEFAULT_TABLE table=test-table [id=b485237a96ce49dda78643664350ce00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:21:16.645812 15593 tablet_service.cc:1467] Processing CreateTablet for tablet b69214f6f73c48e1a814b494426fec52 (DEFAULT_TABLE table=test-table [id=b485237a96ce49dda78643664350ce00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:21:16.645843 15459 tablet_service.cc:1467] Processing CreateTablet for tablet b69214f6f73c48e1a814b494426fec52 (DEFAULT_TABLE table=test-table [id=b485237a96ce49dda78643664350ce00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:21:16.646143 15325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b69214f6f73c48e1a814b494426fec52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:21:16.646143 15593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b69214f6f73c48e1a814b494426fec52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:21:16.646173 15459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b69214f6f73c48e1a814b494426fec52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:21:16.649252 15675 tablet_bootstrap.cc:492] T b69214f6f73c48e1a814b494426fec52 P 518c1405cc8e4a4ab3dcd104cbee0051: Bootstrap starting.
I20260926 06:21:16.649482 15673 tablet_bootstrap.cc:492] T b69214f6f73c48e1a814b494426fec52 P f3d6dc2dbbb240fdbf0c66ce4158aee1: Bootstrap starting.
I20260926 06:21:16.649988 15675 tablet_bootstrap.cc:654] T b69214f6f73c48e1a814b494426fec52 P 518c1405cc8e4a4ab3dcd104cbee0051: Neither blocks nor log segments found. Creating new log.
I20260926 06:21:16.650024 15673 tablet_bootstrap.cc:654] T b69214f6f73c48e1a814b494426fec52 P f3d6dc2dbbb240fdbf0c66ce4158aee1: Neither blocks nor log segments found. Creating new log.
I20260926 06:21:16.650478 15675 log.cc:824] T b69214f6f73c48e1a814b494426fec52 P 518c1405cc8e4a4ab3dcd104cbee0051: Log is configured to *not* fsync() on all Append() calls
I20260926 06:21:16.650501 15673 log.cc:824] T b69214f6f73c48e1a814b494426fec52 P f3d6dc2dbbb240fdbf0c66ce4158aee1: Log is configured to *not* fsync() on all Append() calls
I20260926 06:21:16.650830 15674 tablet_bootstrap.cc:492] T b69214f6f73c48e1a814b494426fec52 P 45852b454b4644a4becf4fef10318127: Bootstrap starting.
I20260926 06:21:16.651230 15675 tablet_bootstrap.cc:492] T b69214f6f73c48e1a814b494426fec52 P 518c1405cc8e4a4ab3dcd104cbee0051: No bootstrap required, opened a new log
I20260926 06:21:16.651233 15673 tablet_bootstrap.cc:492] T b69214f6f73c48e1a814b494426fec52 P f3d6dc2dbbb240fdbf0c66ce4158aee1: No bootstrap required, opened a new log
I20260926 06:21:16.651283 15675 ts_tablet_manager.cc:1397] T b69214f6f73c48e1a814b494426fec52 P 518c1405cc8e4a4ab3dcd104cbee0051: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:21:16.651309 15673 ts_tablet_manager.cc:1397] T b69214f6f73c48e1a814b494426fec52 P f3d6dc2dbbb240fdbf0c66ce4158aee1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:21:16.652798 15673 raft_consensus.cc:348] T b69214f6f73c48e1a814b494426fec52 P f3d6dc2dbbb240fdbf0c66ce4158aee1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d6dc2dbbb240fdbf0c66ce4158aee1" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 37113 } } peers { permanent_uuid: "518c1405cc8e4a4ab3dcd104cbee0051" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40133 } } peers { permanent_uuid: "45852b454b4644a4becf4fef10318127" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 39499 } }
I20260926 06:21:16.652807 15675 raft_consensus.cc:348] T b69214f6f73c48e1a814b494426fec52 P 518c1405cc8e4a4ab3dcd104cbee0051 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d6dc2dbbb240fdbf0c66ce4158aee1" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 37113 } } peers { permanent_uuid: "518c1405cc8e4a4ab3dcd104cbee0051" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40133 } } peers { permanent_uuid: "45852b454b4644a4becf4fef10318127" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 39499 } }
I20260926 06:21:16.652951 15675 raft_consensus.cc:374] T b69214f6f73c48e1a814b494426fec52 P 518c1405cc8e4a4ab3dcd104cbee0051 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:21:16.652953 15673 raft_consensus.cc:374] T b69214f6f73c48e1a814b494426fec52 P f3d6dc2dbbb240fdbf0c66ce4158aee1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:21:16.652971 15675 raft_consensus.cc:729] T b69214f6f73c48e1a814b494426fec52 P 518c1405cc8e4a4ab3dcd104cbee0051 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 518c1405cc8e4a4ab3dcd104cbee0051, State: Initialized, Role: FOLLOWER
I20260926 06:21:16.653000 15673 raft_consensus.cc:729] T b69214f6f73c48e1a814b494426fec52 P f3d6dc2dbbb240fdbf0c66ce4158aee1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d6dc2dbbb240fdbf0c66ce4158aee1, State: Initialized, Role: FOLLOWER
I20260926 06:21:16.653111 15675 consensus_queue.cc:260] T b69214f6f73c48e1a814b494426fec52 P 518c1405cc8e4a4ab3dcd104cbee0051 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d6dc2dbbb240fdbf0c66ce4158aee1" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 37113 } } peers { permanent_uuid: "518c1405cc8e4a4ab3dcd104cbee0051" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40133 } } peers { permanent_uuid: "45852b454b4644a4becf4fef10318127" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 39499 } }
I20260926 06:21:16.653111 15673 consensus_queue.cc:260] T b69214f6f73c48e1a814b494426fec52 P f3d6dc2dbbb240fdbf0c66ce4158aee1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d6dc2dbbb240fdbf0c66ce4158aee1" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 37113 } } peers { permanent_uuid: "518c1405cc8e4a4ab3dcd104cbee0051" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40133 } } peers { permanent_uuid: "45852b454b4644a4becf4fef10318127" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 39499 } }
I20260926 06:21:16.653383 15673 ts_tablet_manager.cc:1428] T b69214f6f73c48e1a814b494426fec52 P f3d6dc2dbbb240fdbf0c66ce4158aee1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:21:16.653383 15675 ts_tablet_manager.cc:1428] T b69214f6f73c48e1a814b494426fec52 P 518c1405cc8e4a4ab3dcd104cbee0051: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:21:16.653422 15525 heartbeater.cc:503] Master 127.4.202.190:44583 was elected leader, sending a full tablet report...
I20260926 06:21:16.653501 15659 heartbeater.cc:503] Master 127.4.202.190:44583 was elected leader, sending a full tablet report...
I20260926 06:21:16.654212 15674 tablet_bootstrap.cc:654] T b69214f6f73c48e1a814b494426fec52 P 45852b454b4644a4becf4fef10318127: Neither blocks nor log segments found. Creating new log.
I20260926 06:21:16.654762 15674 log.cc:824] T b69214f6f73c48e1a814b494426fec52 P 45852b454b4644a4becf4fef10318127: Log is configured to *not* fsync() on all Append() calls
I20260926 06:21:16.655476 15674 tablet_bootstrap.cc:492] T b69214f6f73c48e1a814b494426fec52 P 45852b454b4644a4becf4fef10318127: No bootstrap required, opened a new log
I20260926 06:21:16.655539 15674 ts_tablet_manager.cc:1397] T b69214f6f73c48e1a814b494426fec52 P 45852b454b4644a4becf4fef10318127: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260926 06:21:16.656965 15674 raft_consensus.cc:348] T b69214f6f73c48e1a814b494426fec52 P 45852b454b4644a4becf4fef10318127 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d6dc2dbbb240fdbf0c66ce4158aee1" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 37113 } } peers { permanent_uuid: "518c1405cc8e4a4ab3dcd104cbee0051" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40133 } } peers { permanent_uuid: "45852b454b4644a4becf4fef10318127" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 39499 } }
I20260926 06:21:16.657076 15674 raft_consensus.cc:374] T b69214f6f73c48e1a814b494426fec52 P 45852b454b4644a4becf4fef10318127 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:21:16.657097 15674 raft_consensus.cc:729] T b69214f6f73c48e1a814b494426fec52 P 45852b454b4644a4becf4fef10318127 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45852b454b4644a4becf4fef10318127, State: Initialized, Role: FOLLOWER
I20260926 06:21:16.657214 15674 consensus_queue.cc:260] T b69214f6f73c48e1a814b494426fec52 P 45852b454b4644a4becf4fef10318127 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d6dc2dbbb240fdbf0c66ce4158aee1" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 37113 } } peers { permanent_uuid: "518c1405cc8e4a4ab3dcd104cbee0051" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40133 } } peers { permanent_uuid: "45852b454b4644a4becf4fef10318127" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 39499 } }
I20260926 06:21:16.657490 15391 heartbeater.cc:503] Master 127.4.202.190:44583 was elected leader, sending a full tablet report...
I20260926 06:21:16.657701 15674 ts_tablet_manager.cc:1428] T b69214f6f73c48e1a814b494426fec52 P 45852b454b4644a4becf4fef10318127: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:21:16.658416 15681 raft_consensus.cc:482] T b69214f6f73c48e1a814b494426fec52 P 45852b454b4644a4becf4fef10318127 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:21:16.658489 15681 raft_consensus.cc:504] T b69214f6f73c48e1a814b494426fec52 P 45852b454b4644a4becf4fef10318127 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d6dc2dbbb240fdbf0c66ce4158aee1" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 37113 } } peers { permanent_uuid: "518c1405cc8e4a4ab3dcd104cbee0051" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40133 } } peers { permanent_uuid: "45852b454b4644a4becf4fef10318127" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 39499 } }
I20260926 06:21:16.658799 15681 leader_election.cc:288] T b69214f6f73c48e1a814b494426fec52 P 45852b454b4644a4becf4fef10318127 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d6dc2dbbb240fdbf0c66ce4158aee1 (127.4.202.130:37113), 518c1405cc8e4a4ab3dcd104cbee0051 (127.4.202.131:40133)
I20260926 06:21:16.661641 15670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:16.659126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:55759 (local address 127.4.202.131:40133)
0926 06:21:16.659182 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:16.659184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:16.659191 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:21:16.659211 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:16.659212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:16.659219 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:16.659279 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:21:16.659381 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.659850 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.660898 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.661268 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.661365 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:16.661373 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:21:16.661557 (+ 184us) server_negotiation.cc:299] Negotiation successful
0926 06:21:16.661589 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 06:21:16.661649 15682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:16.658934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:40133 (local address 127.4.202.129:55759)
0926 06:21:16.659077 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:16.659086 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:16.659127 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:16.659275 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:16.659276 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:16.659277 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:21:16.659320 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:16.659321 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.659933 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.659939 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:16.660818 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:16.660820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.661380 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.661381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:16.661451 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:16.661455 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:21:16.661506 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 06:21:16.661545 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":19,"thread_start_us":82,"threads_started":1}
I20260926 06:21:16.661729 15393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:16.658911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:37113 (local address 127.4.202.129:45967)
0926 06:21:16.658972 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:16.658981 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:16.659034 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:16.659314 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:16.659315 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:16.659317 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:21:16.659403 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:16.659404 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.659997 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.659998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:16.660826 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:16.660828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.661449 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.661450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:16.661518 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:16.661520 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:21:16.661562 (+ 42us) client_negotiation.cc:240] Negotiation successful
0926 06:21:16.661591 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":28}
I20260926 06:21:16.661782 15671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:16.658985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:45967 (local address 127.4.202.130:37113)
0926 06:21:16.659023 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:16.659024 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:16.659033 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:21:16.659097 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:16.659098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:16.659107 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:16.659181 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:21:16.659463 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.659906 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.660929 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.661345 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.661413 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:16.661421 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:21:16.661674 (+ 253us) server_negotiation.cc:299] Negotiation successful
0926 06:21:16.661712 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 06:21:16.661958 15613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b69214f6f73c48e1a814b494426fec52" candidate_uuid: "45852b454b4644a4becf4fef10318127" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "518c1405cc8e4a4ab3dcd104cbee0051" is_pre_election: true
I20260926 06:21:16.662062 15613 raft_consensus.cc:2454] T b69214f6f73c48e1a814b494426fec52 P 518c1405cc8e4a4ab3dcd104cbee0051 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45852b454b4644a4becf4fef10318127 in term 0.
I20260926 06:21:16.662040 15479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b69214f6f73c48e1a814b494426fec52" candidate_uuid: "45852b454b4644a4becf4fef10318127" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d6dc2dbbb240fdbf0c66ce4158aee1" is_pre_election: true
I20260926 06:21:16.662160 15479 raft_consensus.cc:2454] T b69214f6f73c48e1a814b494426fec52 P f3d6dc2dbbb240fdbf0c66ce4158aee1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45852b454b4644a4becf4fef10318127 in term 0.
I20260926 06:21:16.662240 15280 leader_election.cc:302] T b69214f6f73c48e1a814b494426fec52 P 45852b454b4644a4becf4fef10318127 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45852b454b4644a4becf4fef10318127, 518c1405cc8e4a4ab3dcd104cbee0051; no voters:
I20260926 06:21:16.662348 15681 raft_consensus.cc:2781] T b69214f6f73c48e1a814b494426fec52 P 45852b454b4644a4becf4fef10318127 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:21:16.662411 15681 raft_consensus.cc:482] T b69214f6f73c48e1a814b494426fec52 P 45852b454b4644a4becf4fef10318127 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:21:16.662432 15681 raft_consensus.cc:3037] T b69214f6f73c48e1a814b494426fec52 P 45852b454b4644a4becf4fef10318127 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:21:16.663043 15681 raft_consensus.cc:504] T b69214f6f73c48e1a814b494426fec52 P 45852b454b4644a4becf4fef10318127 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d6dc2dbbb240fdbf0c66ce4158aee1" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 37113 } } peers { permanent_uuid: "518c1405cc8e4a4ab3dcd104cbee0051" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40133 } } peers { permanent_uuid: "45852b454b4644a4becf4fef10318127" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 39499 } }
I20260926 06:21:16.663177 15681 leader_election.cc:288] T b69214f6f73c48e1a814b494426fec52 P 45852b454b4644a4becf4fef10318127 [CANDIDATE]: Term 1 election: Requested vote from peers f3d6dc2dbbb240fdbf0c66ce4158aee1 (127.4.202.130:37113), 518c1405cc8e4a4ab3dcd104cbee0051 (127.4.202.131:40133)
I20260926 06:21:16.663337 15479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b69214f6f73c48e1a814b494426fec52" candidate_uuid: "45852b454b4644a4becf4fef10318127" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d6dc2dbbb240fdbf0c66ce4158aee1"
I20260926 06:21:16.663367 15613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b69214f6f73c48e1a814b494426fec52" candidate_uuid: "45852b454b4644a4becf4fef10318127" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "518c1405cc8e4a4ab3dcd104cbee0051"
I20260926 06:21:16.663419 15479 raft_consensus.cc:3037] T b69214f6f73c48e1a814b494426fec52 P f3d6dc2dbbb240fdbf0c66ce4158aee1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:21:16.663447 15613 raft_consensus.cc:3037] T b69214f6f73c48e1a814b494426fec52 P 518c1405cc8e4a4ab3dcd104cbee0051 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:21:16.663986 15479 raft_consensus.cc:2454] T b69214f6f73c48e1a814b494426fec52 P f3d6dc2dbbb240fdbf0c66ce4158aee1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45852b454b4644a4becf4fef10318127 in term 1.
I20260926 06:21:16.664141 15613 raft_consensus.cc:2454] T b69214f6f73c48e1a814b494426fec52 P 518c1405cc8e4a4ab3dcd104cbee0051 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45852b454b4644a4becf4fef10318127 in term 1.
I20260926 06:21:16.664157 15278 leader_election.cc:302] T b69214f6f73c48e1a814b494426fec52 P 45852b454b4644a4becf4fef10318127 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45852b454b4644a4becf4fef10318127, f3d6dc2dbbb240fdbf0c66ce4158aee1; no voters:
I20260926 06:21:16.664309 15681 raft_consensus.cc:2781] T b69214f6f73c48e1a814b494426fec52 P 45852b454b4644a4becf4fef10318127 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:21:16.664467 15681 raft_consensus.cc:686] T b69214f6f73c48e1a814b494426fec52 P 45852b454b4644a4becf4fef10318127 [term 1 LEADER]: Becoming Leader. State: Replica: 45852b454b4644a4becf4fef10318127, State: Running, Role: LEADER
I20260926 06:21:16.664561 15681 consensus_queue.cc:237] T b69214f6f73c48e1a814b494426fec52 P 45852b454b4644a4becf4fef10318127 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d6dc2dbbb240fdbf0c66ce4158aee1" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 37113 } } peers { permanent_uuid: "518c1405cc8e4a4ab3dcd104cbee0051" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40133 } } peers { permanent_uuid: "45852b454b4644a4becf4fef10318127" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 39499 } }
I20260926 06:21:16.665308 15200 catalog_manager.cc:5495] T b69214f6f73c48e1a814b494426fec52 P 45852b454b4644a4becf4fef10318127 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45852b454b4644a4becf4fef10318127 (127.4.202.129). New cstate: current_term: 1 leader_uuid: "45852b454b4644a4becf4fef10318127" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d6dc2dbbb240fdbf0c66ce4158aee1" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 37113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "518c1405cc8e4a4ab3dcd104cbee0051" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45852b454b4644a4becf4fef10318127" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 39499 } health_report { overall_health: HEALTHY } } }
I20260926 06:21:16.671746 15672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:16.669303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58582 (local address 127.4.202.129:39499)
0926 06:21:16.669364 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:16.669365 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:16.669374 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:21:16.669444 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:16.669445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:16.669466 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:16.669539 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:21:16.669682 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.670135 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.670992 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:16.671173 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:16.671270 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:16.671448 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:21:16.671525 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:21:16.671600 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 06:21:16.671639 (+ 39us) server_negotiation.cc:299] Negotiation successful
0926 06:21:16.671656 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 06:21:16.673173 15613 raft_consensus.cc:1261] T b69214f6f73c48e1a814b494426fec52 P 518c1405cc8e4a4ab3dcd104cbee0051 [term 1 FOLLOWER]: Refusing update from remote peer 45852b454b4644a4becf4fef10318127: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:21:16.673173 15479 raft_consensus.cc:1261] T b69214f6f73c48e1a814b494426fec52 P f3d6dc2dbbb240fdbf0c66ce4158aee1 [term 1 FOLLOWER]: Refusing update from remote peer 45852b454b4644a4becf4fef10318127: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:21:16.673538 15683 consensus_queue.cc:1035] T b69214f6f73c48e1a814b494426fec52 P 45852b454b4644a4becf4fef10318127 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d6dc2dbbb240fdbf0c66ce4158aee1" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 37113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:21:16.673667 15681 consensus_queue.cc:1035] T b69214f6f73c48e1a814b494426fec52 P 45852b454b4644a4becf4fef10318127 [LEADER]: Connected to new peer: Peer: permanent_uuid: "518c1405cc8e4a4ab3dcd104cbee0051" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:21:16.675649 15686 mvcc.cc:205] Tried to move back new op lower bound from 7333493459651051520 to 7333493459618680832. Current Snapshot: MvccSnapshot[applied={T|T < 7333493459651051520}]
I20260926 06:21:16.676100 15687 mvcc.cc:205] Tried to move back new op lower bound from 7333493459651051520 to 7333493459618680832. Current Snapshot: MvccSnapshot[applied={T|T < 7333493459651051520 or (T in {7333493459651051520})}]
I20260926 06:21:16.676403 15688 mvcc.cc:205] Tried to move back new op lower bound from 7333493459651051520 to 7333493459618680832. Current Snapshot: MvccSnapshot[applied={T|T < 7333493459651051520}]
I20260926 06:21:16.678294 15200 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:21:16.678395 15200 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:21:16.679117 15200 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0e0afceed4334e75841df71fabcb7787: Sending DeleteTablet for 3 replicas of tablet b69214f6f73c48e1a814b494426fec52
I20260926 06:21:16.679406 15459 tablet_service.cc:1514] Processing DeleteTablet for tablet b69214f6f73c48e1a814b494426fec52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:21:16 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:59610
I20260926 06:21:16.679425 15325 tablet_service.cc:1514] Processing DeleteTablet for tablet b69214f6f73c48e1a814b494426fec52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:21:16 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:58568
I20260926 06:21:16.679428 15593 tablet_service.cc:1514] Processing DeleteTablet for tablet b69214f6f73c48e1a814b494426fec52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:21:16 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:46414
I20260926 06:21:16.679627 15696 tablet_replica.cc:330] T b69214f6f73c48e1a814b494426fec52 P f3d6dc2dbbb240fdbf0c66ce4158aee1: stopping tablet replica
I20260926 06:21:16.679723 15696 raft_consensus.cc:2229] T b69214f6f73c48e1a814b494426fec52 P f3d6dc2dbbb240fdbf0c66ce4158aee1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:21:16.679821 15696 raft_consensus.cc:2258] T b69214f6f73c48e1a814b494426fec52 P f3d6dc2dbbb240fdbf0c66ce4158aee1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:21:16.679809 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 15262
I20260926 06:21:16.680298 15697 tablet_replica.cc:330] T b69214f6f73c48e1a814b494426fec52 P 518c1405cc8e4a4ab3dcd104cbee0051: stopping tablet replica
I20260926 06:21:16.680425 15697 raft_consensus.cc:2229] T b69214f6f73c48e1a814b494426fec52 P 518c1405cc8e4a4ab3dcd104cbee0051 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:21:16.680522 15697 raft_consensus.cc:2258] T b69214f6f73c48e1a814b494426fec52 P 518c1405cc8e4a4ab3dcd104cbee0051 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:21:16.680931 15697 ts_tablet_manager.cc:1905] T b69214f6f73c48e1a814b494426fec52 P 518c1405cc8e4a4ab3dcd104cbee0051: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:21:16.682214 15696 ts_tablet_manager.cc:1905] T b69214f6f73c48e1a814b494426fec52 P f3d6dc2dbbb240fdbf0c66ce4158aee1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:21:16.682636 15697 ts_tablet_manager.cc:1918] T b69214f6f73c48e1a814b494426fec52 P 518c1405cc8e4a4ab3dcd104cbee0051: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:21:16.682708 15697 log.cc:1196] T b69214f6f73c48e1a814b494426fec52 P 518c1405cc8e4a4ab3dcd104cbee0051: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-2/wal/wals/b69214f6f73c48e1a814b494426fec52
I20260926 06:21:16.683019 15697 ts_tablet_manager.cc:1939] T b69214f6f73c48e1a814b494426fec52 P 518c1405cc8e4a4ab3dcd104cbee0051: Deleting consensus metadata
I20260926 06:21:16.683403 15186 catalog_manager.cc:4841] TS 518c1405cc8e4a4ab3dcd104cbee0051 (127.4.202.131:40133): tablet b69214f6f73c48e1a814b494426fec52 (table test-table [id=b485237a96ce49dda78643664350ce00]) successfully deleted
I20260926 06:21:16.683822 15696 ts_tablet_manager.cc:1918] T b69214f6f73c48e1a814b494426fec52 P f3d6dc2dbbb240fdbf0c66ce4158aee1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:21:16.683923 15696 log.cc:1196] T b69214f6f73c48e1a814b494426fec52 P f3d6dc2dbbb240fdbf0c66ce4158aee1: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790403591640726-4906-0/minicluster-data/ts-1/wal/wals/b69214f6f73c48e1a814b494426fec52
I20260926 06:21:16.684186 15696 ts_tablet_manager.cc:1939] T b69214f6f73c48e1a814b494426fec52 P f3d6dc2dbbb240fdbf0c66ce4158aee1: Deleting consensus metadata
I20260926 06:21:16.684574 15184 catalog_manager.cc:4841] TS f3d6dc2dbbb240fdbf0c66ce4158aee1 (127.4.202.130:37113): tablet b69214f6f73c48e1a814b494426fec52 (table test-table [id=b485237a96ce49dda78643664350ce00]) successfully deleted
W20260926 06:21:16.685778 15187 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.202.129:39499 (error 108)
I20260926 06:21:16.686234 15668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:16.686114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:39499 (local address 127.0.0.1:58584)
0926 06:21:16.686168 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:16.686205 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.129:39499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260926 06:21:16.686257 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 15396
W20260926 06:21:16.686357 15187 catalog_manager.cc:4584] TS 45852b454b4644a4becf4fef10318127 (127.4.202.129:39499): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b69214f6f73c48e1a814b494426fec52: Network error: Client connection negotiation failed: client connection to 127.4.202.129:39499: connect: Connection refused (error 111)
I20260926 06:21:16.691390 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 15530
I20260926 06:21:16.696436 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 15168
2026-09-26T06:21:16Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1857 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:21:16 dist-test-slave-pd0t krb5kdc[15701](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:21:16 dist-test-slave-pd0t krb5kdc[15701](info): set up 2 sockets
Sep 26 06:21:16 dist-test-slave-pd0t krb5kdc[15701](info): commencing operation
krb5kdc: starting...
W20260926 06:21:17.718645 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.001s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:21:17 dist-test-slave-pd0t krb5kdc[15701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:21:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:21:17Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.4.202.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.4.202.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.4.202.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/rangeradmin_127.4.202.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.4.202.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/rangeradmin_127.4.202.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.4.202.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.4.202.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.4.202.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/rangerlookup_127.4.202.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.4.202.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/rangerlookup_127.4.202.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.202.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.4.202.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.202.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/HTTP_127.4.202.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.202.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/HTTP_127.4.202.148@KRBTEST.COM.keytab.
I20260926 06:21:17.871330 4906 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskm7eEJS/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/postgres -l logfile start
2026-09-26 06:21:20.250 UTC [15731] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-26 06:21:20.250 UTC [15731] LOG: listening on IPv4 address "127.4.202.148", port 52507
2026-09-26 06:21:20.253 UTC [15731] LOG: listening on Unix socket "/tmp/.s.PGSQL.52507"
2026-09-26 06:21:20.274 UTC [15734] LOG: database system was shut down at 2026-09-26 06:21:18 UTC
2026-09-26 06:21:20.277 UTC [15731] LOG: database system is ready to accept connections
I20260926 06:21:21.226567 4906 mini_postgres.cc:96] Postgres bound to 52507
2026-09-26 06:21:21.232 UTC [15742] FATAL: database "slave" does not exist
127.4.202.148:52507 - accepting connections
I20260926 06:21:21.233146 4906 mini_ranger.cc:161] Starting Ranger...
I20260926 06:21:21.247076 4906 mini_ranger.cc:84] Created miniranger Postgres user
I20260926 06:21:21.713536 4906 mini_ranger.cc:87] Created ranger Postgres database
I20260926 06:21:21.713630 4906 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ranger-admin
2026-09-26 06:21:21,981 [I] DB FLAVOR :POSTGRES
2026-09-26 06:21:21,981 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 06:21:21,981 [I] Checking connection..
2026-09-26 06:21:21,982 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:52507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-26 06:21:22,233 [I] Checking connection passed.
2026-09-26 06:21:22,233 [I] --------- Verifying version history table ---------
2026-09-26 06:21:22,234 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:52507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 06:21:22,458 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 06:21:22,458 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:52507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 06:21:22,690 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 06:21:22,690 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-26 06:21:22,690 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:52507/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-26 06:21:22.906 UTC [15806] WARNING: there is no transaction in progress
2026-09-26 06:21:22,912 [I] create_dbversion_catalog.sql file imported successfully
2026-09-26 06:21:22,913 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:52507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 06:21:23,149 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-26 06:21:23,149 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 06:21:23,149 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:52507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-26 06:21:23,365 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:52507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-26 06:21:23,588 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:52507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-pd0t.c.gcp-upstream.internal','N') ;"
2026-09-26 06:21:23,824 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:52507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 06:21:24,056 [I] Table x_portal_user does not exist in database ranger
2026-09-26 06:21:24,056 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:52507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 06:21:24,288 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-26 06:21:24,288 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:52507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 06:21:24,521 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:52507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 06:21:24,760 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-26 06:21:24,760 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:52507/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-26 06:21:25.676 UTC [15914] WARNING: there is no transaction in progress
2026-09-26 06:21:25.692 UTC [15914] WARNING: there is no transaction in progress
2026-09-26 06:21:25.707 UTC [15914] WARNING: there is no transaction in progress
2026-09-26 06:21:25.722 UTC [15914] WARNING: there is no transaction in progress
2026-09-26 06:21:25.738 UTC [15914] WARNING: there is no transaction in progress
2026-09-26 06:21:25.754 UTC [15914] WARNING: there is no transaction in progress
2026-09-26 06:21:25.804 UTC [15914] WARNING: there is no transaction in progress
2026-09-26 06:21:25.810 UTC [15914] WARNING: there is no transaction in progress
2026-09-26 06:21:25.824 UTC [15914] WARNING: there is no transaction in progress
2026-09-26 06:21:25.833 UTC [15914] WARNING: there is no transaction in progress
2026-09-26 06:21:25.843 UTC [15914] WARNING: there is no transaction in progress
2026-09-26 06:21:25.856 UTC [15914] WARNING: there is no transaction in progress
2026-09-26 06:21:25.865 UTC [15914] WARNING: there is no transaction in progress
2026-09-26 06:21:25.871 UTC [15914] WARNING: there is no transaction in progress
2026-09-26 06:21:25.877 UTC [15914] WARNING: there is no transaction in progress
2026-09-26 06:21:25.883 UTC [15914] WARNING: there is no transaction in progress
2026-09-26 06:21:26,494 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-26 06:21:26,495 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:52507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-26 06:21:26,711 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-26 06:21:26,711 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:52507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 06:21:26,956 [I] Table x_portal_user already exists in database 'ranger'
2026-09-26 06:21:26,956 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:52507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 06:21:27,192 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-26 06:21:27,192 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:52507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 06:21:27,407 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:52507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 06:21:27,649 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:52507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-pd0t.c.gcp-upstream.internal';"
2026-09-26 06:21:27,875 [I] CORE_DB_SCHEMA import status has been updated
2026-09-26 06:21:27,875 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:52507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-26 06:21:28,125 [I] DB_PATCHES have already been applied
I20260926 06:21:28.130501 4906 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ranger-admin:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskm7eEJS/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260926 06:21:28.130559 4906 mini_ranger.cc:193] Using host: 127.4.202.148
I20260926 06:21:28.133455 4906 mini_ranger.cc:227] Ranger admin URL: http://127.4.202.148:45719
Sep 26, 2026 6:21:28 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 6:21:28 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 6:21:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ranger-admin/ews/webapp
Sep 26, 2026 6:21:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 26, 2026 6:21:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 26, 2026 6:21:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ranger-admin/ews/webapp].
Sep 26, 2026 6:21:28 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260926 06:21:29.234472 4906 mini_ranger.cc:160] Time spent starting Ranger: real 8.001s user 0.000s sys 0.009s
Sep 26, 2026 6:21:47 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 26, 2026 6:21:47 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 26, 2026 6:21:47 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 26, 2026 6:21:47 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 26, 2026 6:21:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 26, 2026 6:21:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 26, 2026 6:21:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 26, 2026 6:21:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 26, 2026 6:21:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 26, 2026 6:21:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 26, 2026 6:21:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 26, 2026 6:21:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 26, 2026 6:21:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 26, 2026 6:21:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 26, 2026 6:21:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 26, 2026 6:21:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 26, 2026 6:21:47 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 26, 2026 6:21:48 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260926 06:21:49.219105 4906 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.4.202.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.4.202.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.4.202.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/rangerkms_127.4.202.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.4.202.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/rangerkms_127.4.202.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.202.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.4.202.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.202.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/HTTP_127.4.202.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.202.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/HTTP_127.4.202.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 26 06:21:49 dist-test-slave-pd0t krb5kdc[15701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403709, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260926 06:21:49.296964 4906 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260926 06:21:49.305691 4906 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260926 06:21:50.169692 4906 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260926 06:21:50.169776 4906 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ranger-kms
I20260926 06:21:50.169798 4906 mini_ranger_kms.cc:227] Using postgres at 127.4.202.148:52507
2026-09-26 06:21:50,346 [I] DB FLAVOR :POSTGRES
2026-09-26 06:21:50,347 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 06:21:50,347 [I] Checking connection
2026-09-26 06:21:50,347 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:52507/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-26 06:21:50,590 [I] connection success
2026-09-26 06:21:50,590 [I] --------- Verifying Ranger DB tables ---------
2026-09-26 06:21:50,590 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-26 06:21:50,590 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:52507/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-26 06:21:50,810 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-26 06:21:50,810 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 06:21:50,810 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-26 06:21:50,810 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:52507/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-26 06:21:51,051 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260926 06:21:51.461041 4906 mini_ranger_kms.cc:310] Created kms service
I20260926 06:21:51.659291 4906 mini_ranger_kms.cc:326] Created kudu user
I20260926 06:21:51.700282 4906 mini_ranger_kms.cc:343] Created rangerkms user
Sep 26 06:21:51 dist-test-slave-pd0t krb5kdc[15701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403709, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.4.202.148@KRBTEST.COM
I20260926 06:21:52.079639 4906 mini_ranger_kms.cc:384] Added ranger policy
I20260926 06:21:52.079747 4906 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ranger-kms:/tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260926 06:21:52.079770 4906 mini_ranger_kms.cc:240] Using host: 127.4.202.148
I20260926 06:21:52.083007 4906 mini_ranger_kms.cc:276] Ranger KMS PID: 16094
I20260926 06:21:52.083146 4906 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.4.202.148:46085
Sep 26, 2026 6:21:53 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 6:21:53 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 6:21:53 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 26, 2026 6:21:53 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 26, 2026 6:21:53 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ranger-kms/ews/webapp].
Sep 26, 2026 6:21:53 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260926 06:21:54.286760 4906 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.990s user 0.010s sys 0.151s
I20260926 06:21:54.286887 4906 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260926 06:21:54.286973 4906 mini_ranger_kms.cc:401] 127.4.202.148:46085/kms/v1/keys
Sep 26 06:21:54 dist-test-slave-pd0t krb5kdc[15701](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790403714, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.202.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 26 06:21:54 dist-test-slave-pd0t krb5kdc[15701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403714, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.202.148@KRBTEST.COM for HTTP/127.4.202.148@KRBTEST.COM
Sep 26 06:21:55 dist-test-slave-pd0t krb5kdc[15701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403714, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.202.148@KRBTEST.COM for HTTP/127.4.202.148@KRBTEST.COM
Sep 26 06:21:57 dist-test-slave-pd0t krb5kdc[15701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.4.202.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
I20260926 06:21:57.385926 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:39563
--webserver_interface=127.4.202.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:46211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:39563
--ranger_config_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.202.148:46085/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:21:57.464583 16145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:21:57.464792 16145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:21:57.464813 16145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:21:57.465927 16145 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:21:57.465982 16145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:21:57.465996 16145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:21:57.466008 16145 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:21:57.466020 16145 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:21:57.467505 16145 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:46211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/master-0/wal
--ranger_kms_url=127.4.202.148:46085/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:39563
--ranger_config_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:39563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:21:57.467744 16145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:21:57.467971 16145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:21:57.472652 16150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:21:57.472652 16153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:21:57.472675 16151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:21:57.472944 16145 server_base.cc:1034] running on GCE node
I20260926 06:21:57.473172 16145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:21:57.473483 16145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:21:57.474663 16145 hybrid_clock.cc:648] HybridClock initialized: now 1790403717474632 us; error 41 us; skew 500 ppm
Sep 26 06:21:57 dist-test-slave-pd0t krb5kdc[15701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403717, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:21:57.481894 16145 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:21:57.482328 16145 webserver.cc:460] Webserver started at http://127.4.202.190:34447/ using document root <none> and password file <none>
I20260926 06:21:57.482805 16145 fs_manager.cc:359] Metadata directory not provided
I20260926 06:21:57.482851 16145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:21:57.482941 16145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 06:21:57 dist-test-slave-pd0t krb5kdc[15701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403717, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for HTTP/127.4.202.148@KRBTEST.COM
I20260926 06:21:57.534379 16145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "42059e997cd347c58681fb95bd2bc314"
format_stamp: "Formatted at 2026-09-26 06:21:57 on dist-test-slave-pd0t"
server_key: "f5120f6f718ab0eda967309e1b185a67"
server_key_iv: "0eba46902e352a147bb76de58c8bc424"
server_key_version: "kuduclusterkey@0"
I20260926 06:21:57.534821 16145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "42059e997cd347c58681fb95bd2bc314"
format_stamp: "Formatted at 2026-09-26 06:21:57 on dist-test-slave-pd0t"
server_key: "f5120f6f718ab0eda967309e1b185a67"
server_key_iv: "0eba46902e352a147bb76de58c8bc424"
server_key_version: "kuduclusterkey@0"
I20260926 06:21:57.536185 16145 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:21:57.548863 16160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:57.549105 16145 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:21:57.549196 16145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "42059e997cd347c58681fb95bd2bc314"
format_stamp: "Formatted at 2026-09-26 06:21:57 on dist-test-slave-pd0t"
server_key: "f5120f6f718ab0eda967309e1b185a67"
server_key_iv: "0eba46902e352a147bb76de58c8bc424"
server_key_version: "kuduclusterkey@0"
I20260926 06:21:57.549278 16145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:21:57.560607 16145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:21:57.562471 16145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:21:57.562629 16145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:21:57.567729 16145 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:39563
I20260926 06:21:57.567754 16212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:39563 every 8 connection(s)
I20260926 06:21:57.568077 16145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:21:57.568722 16213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:21:57.570868 16213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42059e997cd347c58681fb95bd2bc314: Bootstrap starting.
I20260926 06:21:57.571070 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 16145
I20260926 06:21:57.571171 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:21:57.571357 4906 external_mini_cluster.cc:1428] Setting key df3825455ba09ac7834d1ab43132704d
I20260926 06:21:57.571558 16213 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42059e997cd347c58681fb95bd2bc314: Neither blocks nor log segments found. Creating new log.
I20260926 06:21:57.571885 16213 log.cc:824] T 00000000000000000000000000000000 P 42059e997cd347c58681fb95bd2bc314: Log is configured to *not* fsync() on all Append() calls
I20260926 06:21:57.572597 16213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42059e997cd347c58681fb95bd2bc314: No bootstrap required, opened a new log
I20260926 06:21:57.574312 16213 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42059e997cd347c58681fb95bd2bc314 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42059e997cd347c58681fb95bd2bc314" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 39563 } }
I20260926 06:21:57.574465 16213 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42059e997cd347c58681fb95bd2bc314 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:21:57.574507 16213 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42059e997cd347c58681fb95bd2bc314 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42059e997cd347c58681fb95bd2bc314, State: Initialized, Role: FOLLOWER
I20260926 06:21:57.574641 16213 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42059e997cd347c58681fb95bd2bc314 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42059e997cd347c58681fb95bd2bc314" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 39563 } }
I20260926 06:21:57.574743 16213 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42059e997cd347c58681fb95bd2bc314 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:21:57.574776 16213 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42059e997cd347c58681fb95bd2bc314 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:21:57.574815 16213 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42059e997cd347c58681fb95bd2bc314 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:21:57.575311 16213 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42059e997cd347c58681fb95bd2bc314 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42059e997cd347c58681fb95bd2bc314" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 39563 } }
I20260926 06:21:57.575425 16213 leader_election.cc:302] T 00000000000000000000000000000000 P 42059e997cd347c58681fb95bd2bc314 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42059e997cd347c58681fb95bd2bc314; no voters:
I20260926 06:21:57.575587 16213 leader_election.cc:288] T 00000000000000000000000000000000 P 42059e997cd347c58681fb95bd2bc314 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:21:57.575654 16218 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42059e997cd347c58681fb95bd2bc314 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:21:57.575857 16218 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42059e997cd347c58681fb95bd2bc314 [term 1 LEADER]: Becoming Leader. State: Replica: 42059e997cd347c58681fb95bd2bc314, State: Running, Role: LEADER
I20260926 06:21:57.575881 16213 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42059e997cd347c58681fb95bd2bc314 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:21:57.576026 16218 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42059e997cd347c58681fb95bd2bc314 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42059e997cd347c58681fb95bd2bc314" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 39563 } }
I20260926 06:21:57.576306 16218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42059e997cd347c58681fb95bd2bc314 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42059e997cd347c58681fb95bd2bc314" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42059e997cd347c58681fb95bd2bc314" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 39563 } } }
I20260926 06:21:57.576365 16218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42059e997cd347c58681fb95bd2bc314 [sys.catalog]: This master's current role is: LEADER
I20260926 06:21:57.576484 16218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42059e997cd347c58681fb95bd2bc314 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42059e997cd347c58681fb95bd2bc314. Latest consensus state: current_term: 1 leader_uuid: "42059e997cd347c58681fb95bd2bc314" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42059e997cd347c58681fb95bd2bc314" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 39563 } } }
I20260926 06:21:57.576529 16218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42059e997cd347c58681fb95bd2bc314 [sys.catalog]: This master's current role is: LEADER
I20260926 06:21:57.576936 16225 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:21:57.577315 16213 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Sep 26 06:21:57 dist-test-slave-pd0t krb5kdc[15701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:21:57.582420 16225 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:21:57.584707 16225 catalog_manager.cc:1347] Generated new cluster ID: d354d2740ac14bada2314f5d5e1e2c3d
I20260926 06:21:57.584766 16225 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:21:57.586242 16216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:57.572002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42592 (local address 127.4.202.190:39563)
0926 06:21:57.572175 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:57.572185 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:57.572217 (+ 32us) server_negotiation.cc:407] Connection header received
0926 06:21:57.572755 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:57.572764 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:57.572831 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:57.572991 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:57.573752 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:57.575506 (+ 1754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:57.576590 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:57.577254 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:57.577354 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:57.581362 (+ 4008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:21:57.581412 (+ 50us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:21:57.581413 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:21:57.581500 (+ 87us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:21:57.584271 (+ 2771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:21:57.584702 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:21:57.584705 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:21:57.584705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:21:57.584872 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:21:57.585085 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:21:57.585085 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:21:57.585086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:21:57.585349 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:21:57.585855 (+ 506us) server_negotiation.cc:1035] Waiting for connection context
0926 06:21:57.585918 (+ 63us) server_negotiation.cc:299] Negotiation successful
0926 06:21:57.586042 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":76,"threads_started":1}
I20260926 06:21:57.597273 16225 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:21:57.598047 16225 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:21:57.606979 16225 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42059e997cd347c58681fb95bd2bc314: Generated new TSK 0
I20260926 06:21:57.607192 16225 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.202.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
I20260926 06:21:58.262256 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.202.148:46085/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.129:0
--local_ip_for_outbound_sockets=127.4.202.129
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:39563
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:46211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:21:58.364914 16258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:21:58.365185 16258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:21:58.365217 16258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:21:58.366848 16258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:21:58.366900 16258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:21:58.366936 16258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.129
I20260926 06:21:58.368975 16258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:46211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.4.202.148:46085/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:39563
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.129
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:21:58.369228 16258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:21:58.369486 16258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:21:58.375586 16264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:21:58.375746 16266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:21:58.375982 16263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:21:58.376057 16258 server_base.cc:1034] running on GCE node
I20260926 06:21:58.376341 16258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:21:58.376647 16258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:21:58.377827 16258 hybrid_clock.cc:648] HybridClock initialized: now 1790403718377809 us; error 33 us; skew 500 ppm
Sep 26 06:21:58 dist-test-slave-pd0t krb5kdc[15701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403718, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:21:58.381012 16258 init.cc:377] Logged in from keytab as kudu/127.4.202.129@KRBTEST.COM (short username kudu)
I20260926 06:21:58.381520 16258 webserver.cc:460] Webserver started at http://127.4.202.129:38661/ using document root <none> and password file <none>
I20260926 06:21:58.381781 16258 fs_manager.cc:359] Metadata directory not provided
I20260926 06:21:58.381862 16258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:21:58.382002 16258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 06:21:58 dist-test-slave-pd0t krb5kdc[15701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403718, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for HTTP/127.4.202.148@KRBTEST.COM
I20260926 06:21:58.395835 16258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-0/data/instance:
uuid: "97d7edd439a74f5aa477f951e389cb8a"
format_stamp: "Formatted at 2026-09-26 06:21:58 on dist-test-slave-pd0t"
server_key: "9efce446da376fcd1b74d7b48898bde1"
server_key_iv: "27d1e71723d297bcc2f091e30aeffaf7"
server_key_version: "kuduclusterkey@0"
I20260926 06:21:58.396265 16258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance:
uuid: "97d7edd439a74f5aa477f951e389cb8a"
format_stamp: "Formatted at 2026-09-26 06:21:58 on dist-test-slave-pd0t"
server_key: "9efce446da376fcd1b74d7b48898bde1"
server_key_iv: "27d1e71723d297bcc2f091e30aeffaf7"
server_key_version: "kuduclusterkey@0"
I20260926 06:21:58.407011 16258 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.003s sys 0.000s
I20260926 06:21:58.421778 16275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:58.421954 16258 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 06:21:58.422062 16258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-0/wal
uuid: "97d7edd439a74f5aa477f951e389cb8a"
format_stamp: "Formatted at 2026-09-26 06:21:58 on dist-test-slave-pd0t"
server_key: "9efce446da376fcd1b74d7b48898bde1"
server_key_iv: "27d1e71723d297bcc2f091e30aeffaf7"
server_key_version: "kuduclusterkey@0"
I20260926 06:21:58.422160 16258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:21:58.442576 16258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:21:58.444882 16258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:21:58.445113 16258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:21:58.445358 16258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:21:58.445744 16258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:21:58.445797 16258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:58.445832 16258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:21:58.445858 16258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:58.453600 16258 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.129:37227
I20260926 06:21:58.453680 16388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.129:37227 every 8 connection(s)
I20260926 06:21:58.453991 16258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
I20260926 06:21:58.457115 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 16258
I20260926 06:21:58.457221 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance
I20260926 06:21:58.457407 4906 external_mini_cluster.cc:1428] Setting key b4d6ce6cf01d45e7315efd9ea2b297cb
Sep 26 06:21:58 dist-test-slave-pd0t krb5kdc[15701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403718, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:21:58.465029 16392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:58.454730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:39563 (local address 127.4.202.129:50497)
0926 06:21:58.454952 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:58.454981 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:58.455538 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:58.455803 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:58.455807 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:58.456043 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:58.456297 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:58.456304 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:58.457311 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:58.457312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:58.458545 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:58.458548 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:58.459157 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:58.459158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:58.459295 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:58.459871 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:58.459881 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:58.461477 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:21:58.463470 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:21:58.463476 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:21:58.463478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:21:58.463755 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:21:58.463969 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:21:58.463969 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:21:58.463969 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:21:58.464096 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:21:58.464382 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:21:58.464387 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:21:58.464551 (+ 164us) client_negotiation.cc:769] Sending connection context
0926 06:21:58.464653 (+ 102us) client_negotiation.cc:240] Negotiation successful
0926 06:21:58.464797 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":66,"threads_started":1}
I20260926 06:21:58.465513 16391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:58.454700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:50497 (local address 127.4.202.190:39563)
0926 06:21:58.454879 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:58.454881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:58.455057 (+ 176us) server_negotiation.cc:407] Connection header received
0926 06:21:58.455635 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:58.455637 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:58.455651 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:58.455734 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:58.456399 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:58.457199 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:58.458703 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:58.459033 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:58.459093 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:58.461581 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:21:58.461582 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:21:58.461583 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:21:58.461590 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:21:58.463378 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:21:58.463839 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:21:58.463840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:21:58.463840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:21:58.463903 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:21:58.464166 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:21:58.464167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:21:58.464167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:21:58.464299 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:21:58.464357 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 06:21:58.465340 (+ 983us) server_negotiation.cc:299] Negotiation successful
0926 06:21:58.465418 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":95,"threads_started":1}
I20260926 06:21:58.466200 16389 heartbeater.cc:347] Connected to a master server at 127.4.202.190:39563
I20260926 06:21:58.466303 16389 heartbeater.cc:464] Registering TS with master...
I20260926 06:21:58.466528 16389 heartbeater.cc:511] Master 127.4.202.190:39563 requested a full tablet report, sending...
I20260926 06:21:58.467059 16177 ts_manager.cc:194] Registered new tserver with Master: 97d7edd439a74f5aa477f951e389cb8a (127.4.202.129:37227)
I20260926 06:21:58.467911 16177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.129@KRBTEST.COM'} at 127.4.202.129:50497
WARNING: no policy specified for kudu/127.4.202.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
I20260926 06:21:58.490119 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.202.148:46085/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.130:0
--local_ip_for_outbound_sockets=127.4.202.130
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:39563
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:46211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 06:21:58 dist-test-slave-pd0t krb5kdc[15701](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790403718, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260926 06:21:58.590981 16395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:21:58.591286 16395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:21:58.591382 16395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:21:58.592558 16395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:21:58.592625 16395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:21:58.592698 16395 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.130
I20260926 06:21:58.594779 16395 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:46211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.4.202.148:46085/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:39563
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.130
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:21:58.595217 16395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:21:58.595504 16395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:21:58.600852 16405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:21:58.601218 16403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:21:58.601567 16402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:21:58.602794 16395 server_base.cc:1034] running on GCE node
I20260926 06:21:58.603021 16395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:21:58.603288 16395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:21:58.604449 16395 hybrid_clock.cc:648] HybridClock initialized: now 1790403718604428 us; error 35 us; skew 500 ppm
Sep 26 06:21:58 dist-test-slave-pd0t krb5kdc[15701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403718, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:21:58.607720 16395 init.cc:377] Logged in from keytab as kudu/127.4.202.130@KRBTEST.COM (short username kudu)
I20260926 06:21:58.608207 16395 webserver.cc:460] Webserver started at http://127.4.202.130:37449/ using document root <none> and password file <none>
I20260926 06:21:58.608460 16395 fs_manager.cc:359] Metadata directory not provided
I20260926 06:21:58.608522 16395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:21:58.608690 16395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 06:21:58 dist-test-slave-pd0t krb5kdc[15701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403718, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for HTTP/127.4.202.148@KRBTEST.COM
I20260926 06:21:58.624060 16395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-1/data/instance:
uuid: "4341265376174d3e96019a104fc3da7f"
format_stamp: "Formatted at 2026-09-26 06:21:58 on dist-test-slave-pd0t"
server_key: "4779a0ac50fe0c08e63980a710098e5d"
server_key_iv: "fcf1c1053037c7efb9ab83f9ef62fd7c"
server_key_version: "kuduclusterkey@0"
I20260926 06:21:58.624475 16395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance:
uuid: "4341265376174d3e96019a104fc3da7f"
format_stamp: "Formatted at 2026-09-26 06:21:58 on dist-test-slave-pd0t"
server_key: "4779a0ac50fe0c08e63980a710098e5d"
server_key_iv: "fcf1c1053037c7efb9ab83f9ef62fd7c"
server_key_version: "kuduclusterkey@0"
I20260926 06:21:58.628175 16395 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.000s
I20260926 06:21:58.639225 16412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:58.639441 16395 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:21:58.639526 16395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-1/wal
uuid: "4341265376174d3e96019a104fc3da7f"
format_stamp: "Formatted at 2026-09-26 06:21:58 on dist-test-slave-pd0t"
server_key: "4779a0ac50fe0c08e63980a710098e5d"
server_key_iv: "fcf1c1053037c7efb9ab83f9ef62fd7c"
server_key_version: "kuduclusterkey@0"
I20260926 06:21:58.639601 16395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:21:58.652098 16395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:21:58.654801 16395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:21:58.654980 16395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:21:58.655200 16395 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:21:58.655581 16395 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:21:58.655669 16395 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:58.655742 16395 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:21:58.655800 16395 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:58.662410 16395 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.130:39919
I20260926 06:21:58.662716 16526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.130:39919 every 8 connection(s)
I20260926 06:21:58.663133 16395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
I20260926 06:21:58.664579 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 16395
I20260926 06:21:58.664681 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance
I20260926 06:21:58.664867 4906 external_mini_cluster.cc:1428] Setting key 6d538a867ad42622cc13aa8d3a23a477
Sep 26 06:21:58 dist-test-slave-pd0t krb5kdc[15701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403718, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:21:58.674232 16391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:58.664082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:33059 (local address 127.4.202.190:39563)
0926 06:21:58.664130 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:58.664133 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:58.664181 (+ 48us) server_negotiation.cc:407] Connection header received
0926 06:21:58.664943 (+ 762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:58.664945 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:58.664963 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:58.664995 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:58.665821 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:58.666623 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:58.668170 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:58.668470 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:58.668511 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:58.670992 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:21:58.670994 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:21:58.670994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:21:58.671001 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:21:58.672709 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:21:58.673192 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:21:58.673193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:21:58.673194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:21:58.673248 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:21:58.673506 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:21:58.673507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:21:58.673507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:21:58.673630 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:21:58.673672 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0926 06:21:58.674060 (+ 388us) server_negotiation.cc:299] Negotiation successful
0926 06:21:58.674142 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 06:21:58.674957 16529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:58.663849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:39563 (local address 127.4.202.130:33059)
0926 06:21:58.664043 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:58.664090 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:58.664806 (+ 716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:58.665201 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:58.665205 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:58.665428 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:58.665705 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:58.665712 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:58.666733 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:58.666734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:58.668066 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:58.668069 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:58.668599 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:58.668600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:58.668739 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:58.669287 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:58.669296 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:58.670888 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:21:58.672816 (+ 1928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:21:58.672822 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:21:58.672824 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:21:58.673104 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:21:58.673316 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:21:58.673317 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:21:58.673317 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:21:58.673438 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:21:58.673716 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:21:58.673721 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:21:58.673903 (+ 182us) client_negotiation.cc:769] Sending connection context
0926 06:21:58.674564 (+ 661us) client_negotiation.cc:240] Negotiation successful
0926 06:21:58.674743 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":84,"threads_started":1}
I20260926 06:21:58.675588 16527 heartbeater.cc:347] Connected to a master server at 127.4.202.190:39563
I20260926 06:21:58.675670 16527 heartbeater.cc:464] Registering TS with master...
I20260926 06:21:58.675827 16527 heartbeater.cc:511] Master 127.4.202.190:39563 requested a full tablet report, sending...
I20260926 06:21:58.676244 16177 ts_manager.cc:194] Registered new tserver with Master: 4341265376174d3e96019a104fc3da7f (127.4.202.130:39919)
I20260926 06:21:58.676731 16177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.130@KRBTEST.COM'} at 127.4.202.130:33059
WARNING: no policy specified for kudu/127.4.202.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
I20260926 06:21:58.713828 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.202.148:46085/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.131:0
--local_ip_for_outbound_sockets=127.4.202.131
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:39563
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:46211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:21:58.834285 16532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:21:58.834632 16532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:21:58.834738 16532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:21:58.836448 16532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:21:58.836556 16532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:21:58.836740 16532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.131
I20260926 06:21:58.838920 16532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:46211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.4.202.148:46085/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:39563
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.131
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:21:58.839395 16532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:21:58.839700 16532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:21:58.845919 16538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:21:58.845919 16540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:21:58.847226 16532 server_base.cc:1034] running on GCE node
W20260926 06:21:58.848255 16537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:21:58.848534 16532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:21:58.849221 16532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:21:58.850386 16532 hybrid_clock.cc:648] HybridClock initialized: now 1790403718850368 us; error 36 us; skew 500 ppm
Sep 26 06:21:58 dist-test-slave-pd0t krb5kdc[15701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403718, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:21:58.853673 16532 init.cc:377] Logged in from keytab as kudu/127.4.202.131@KRBTEST.COM (short username kudu)
I20260926 06:21:58.854189 16532 webserver.cc:460] Webserver started at http://127.4.202.131:46533/ using document root <none> and password file <none>
I20260926 06:21:58.854427 16532 fs_manager.cc:359] Metadata directory not provided
I20260926 06:21:58.854477 16532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:21:58.854614 16532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 06:21:58 dist-test-slave-pd0t krb5kdc[15701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403718, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for HTTP/127.4.202.148@KRBTEST.COM
I20260926 06:21:58.869138 16532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-2/data/instance:
uuid: "a1f030d3416040ffb9a09038988199e1"
format_stamp: "Formatted at 2026-09-26 06:21:58 on dist-test-slave-pd0t"
server_key: "5ad409f93677c799e1b6ce0ede2ec035"
server_key_iv: "bc246d50952abbdd3e52a477b02c1712"
server_key_version: "kuduclusterkey@0"
I20260926 06:21:58.869618 16532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance:
uuid: "a1f030d3416040ffb9a09038988199e1"
format_stamp: "Formatted at 2026-09-26 06:21:58 on dist-test-slave-pd0t"
server_key: "5ad409f93677c799e1b6ce0ede2ec035"
server_key_iv: "bc246d50952abbdd3e52a477b02c1712"
server_key_version: "kuduclusterkey@0"
I20260926 06:21:58.871361 16532 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260926 06:21:58.884436 16547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:58.884697 16532 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:21:58.884802 16532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-2/wal
uuid: "a1f030d3416040ffb9a09038988199e1"
format_stamp: "Formatted at 2026-09-26 06:21:58 on dist-test-slave-pd0t"
server_key: "5ad409f93677c799e1b6ce0ede2ec035"
server_key_iv: "bc246d50952abbdd3e52a477b02c1712"
server_key_version: "kuduclusterkey@0"
I20260926 06:21:58.884893 16532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:21:58.916543 16532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:21:58.918979 16532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:21:58.919176 16532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:21:58.919458 16532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:21:58.919886 16532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:21:58.919986 16532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:58.920060 16532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:21:58.920120 16532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:21:58.926182 16532 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.131:42621
I20260926 06:21:58.926635 16532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
I20260926 06:21:58.926951 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 16532
I20260926 06:21:58.927086 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance
I20260926 06:21:58.927320 4906 external_mini_cluster.cc:1428] Setting key 70fe23d31c5dedb3cb9ce424f404ea1f
I20260926 06:21:58.928810 16660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.131:42621 every 8 connection(s)
Sep 26 06:21:58 dist-test-slave-pd0t krb5kdc[15701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403718, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:21:58.941182 16391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:58.930297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:54437 (local address 127.4.202.190:39563)
0926 06:21:58.930338 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:58.930341 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:58.931291 (+ 950us) server_negotiation.cc:407] Connection header received
0926 06:21:58.931921 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:58.931922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:58.931933 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:58.931962 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:58.932756 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:58.933528 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:58.935059 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:58.935358 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:58.935400 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:58.937884 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:21:58.937886 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:21:58.937886 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:21:58.937893 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:21:58.939604 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:21:58.940092 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:21:58.940093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:21:58.940094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:21:58.940151 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:21:58.940434 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:21:58.940435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:21:58.940435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:21:58.940563 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:21:58.940607 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0926 06:21:58.941011 (+ 404us) server_negotiation.cc:299] Negotiation successful
0926 06:21:58.941091 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 06:21:58.941766 16663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:58.929610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:39563 (local address 127.4.202.131:54437)
0926 06:21:58.931167 (+ 1557us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:58.931213 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:58.931797 (+ 584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:58.932133 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:58.932137 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:58.932400 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:21:58.932652 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:58.932658 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:58.933644 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:58.933645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:58.934957 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:58.934960 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:58.935485 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:58.935486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:58.935616 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:58.936163 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:21:58.936173 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:21:58.937759 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:21:58.939700 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:21:58.939707 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:21:58.939709 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:21:58.939996 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:21:58.940225 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:21:58.940226 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:21:58.940226 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:21:58.940351 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:21:58.940668 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:21:58.940674 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:21:58.940847 (+ 173us) client_negotiation.cc:769] Sending connection context
0926 06:21:58.941367 (+ 520us) client_negotiation.cc:240] Negotiation successful
0926 06:21:58.941514 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1514,"thread_start_us":513,"threads_started":1}
I20260926 06:21:58.942389 16661 heartbeater.cc:347] Connected to a master server at 127.4.202.190:39563
I20260926 06:21:58.942477 16661 heartbeater.cc:464] Registering TS with master...
I20260926 06:21:58.942731 16661 heartbeater.cc:511] Master 127.4.202.190:39563 requested a full tablet report, sending...
I20260926 06:21:58.943105 16177 ts_manager.cc:194] Registered new tserver with Master: a1f030d3416040ffb9a09038988199e1 (127.4.202.131:42621)
I20260926 06:21:58.943574 16177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.131@KRBTEST.COM'} at 127.4.202.131:54437
I20260926 06:21:58.950242 4906 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:21:58.963268 16391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:58.952455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42596 (local address 127.4.202.190:39563)
0926 06:21:58.952494 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:58.952496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:58.952791 (+ 295us) server_negotiation.cc:407] Connection header received
0926 06:21:58.952906 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:58.952907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:58.952931 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:58.952961 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:21:58.953977 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:58.954748 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:58.955822 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:58.956111 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:58.956151 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:58.957604 (+ 1453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:21:58.957606 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:21:58.957606 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:21:58.957613 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:21:58.960517 (+ 2904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:21:58.962392 (+ 1875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:21:58.962393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:21:58.962393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:21:58.962452 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:21:58.962675 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:21:58.962676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:21:58.962676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:21:58.962800 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:21:58.962882 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 06:21:58.963089 (+ 207us) server_negotiation.cc:299] Negotiation successful
0926 06:21:58.963167 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 06:21:58.965479 16177 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42596:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:21:58.967751 16177 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:21:58.982743 16677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:58.971441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:39919 (local address 127.0.0.1:47454)
0926 06:21:58.972812 (+ 1371us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:58.972863 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:58.972925 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:58.973562 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:58.973565 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:58.973567 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:21:58.973631 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:58.973633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:58.975186 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:58.975187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:58.978922 (+ 3735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:58.978925 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:58.982392 (+ 3467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:58.982393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:58.982476 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:58.982491 (+ 15us) client_negotiation.cc:769] Sending connection context
0926 06:21:58.982563 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 06:21:58.982668 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1340,"thread_start_us":67,"threads_started":1}
I20260926 06:21:58.984045 16678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:58.972114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47454 (local address 127.4.202.130:39919)
0926 06:21:58.973063 (+ 949us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:58.973067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:58.973077 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:21:58.973115 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:58.973117 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:58.973133 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:58.973487 (+ 354us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:21:58.973706 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:58.975091 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:58.979039 (+ 3948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:58.981923 (+ 2884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:58.982270 (+ 347us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:58.982355 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 06:21:58.983920 (+ 1565us) server_negotiation.cc:299] Negotiation successful
0926 06:21:58.983960 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":913,"thread_start_us":74,"threads_started":1}
I20260926 06:21:58.985606 16675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:58.971802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:42621 (local address 127.0.0.1:39182)
0926 06:21:58.973367 (+ 1565us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:58.973374 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:58.973407 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:58.976890 (+ 3483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:58.976891 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:58.976894 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:21:58.976940 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:58.976941 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:58.983161 (+ 6220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:58.983162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:58.984310 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:58.984312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:58.985381 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:58.985382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:58.985444 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:58.985449 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 06:21:58.985496 (+ 47us) client_negotiation.cc:240] Negotiation successful
0926 06:21:58.985540 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1553,"thread_start_us":90,"threads_started":1}
I20260926 06:21:58.985674 16460 tablet_service.cc:1467] Processing CreateTablet for tablet e4236b92d1de45a9897b60ccc4edc377 (DEFAULT_TABLE table=test-table [id=7c9a39aa8f9c41ccad066ce490986717]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:21:58.986048 16460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4236b92d1de45a9897b60ccc4edc377. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:21:58.986074 16679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:58.972294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39182 (local address 127.4.202.131:42621)
0926 06:21:58.976650 (+ 4356us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:58.976655 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:58.976666 (+ 11us) server_negotiation.cc:407] Connection header received
0926 06:21:58.976703 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:58.976705 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:58.976721 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:58.976804 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:21:58.977016 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:58.983065 (+ 6049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:58.984437 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:58.985193 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:58.985264 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:58.985333 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 06:21:58.985949 (+ 616us) server_negotiation.cc:299] Negotiation successful
0926 06:21:58.985996 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4318,"thread_start_us":96,"threads_started":1}
I20260926 06:21:58.986650 16580 tablet_service.cc:1467] Processing CreateTablet for tablet e4236b92d1de45a9897b60ccc4edc377 (DEFAULT_TABLE table=test-table [id=7c9a39aa8f9c41ccad066ce490986717]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:21:58.986956 16580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4236b92d1de45a9897b60ccc4edc377. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:21:58.990253 16674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:58.971643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:37227 (local address 127.0.0.1:41120)
0926 06:21:58.973252 (+ 1609us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:58.973260 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:58.973299 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:58.985253 (+ 11954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:58.985254 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:58.985256 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:21:58.985294 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:58.985296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:58.987998 (+ 2702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:58.987999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:58.989111 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:58.989115 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:58.990045 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:58.990046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:58.990107 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:58.990111 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:21:58.990144 (+ 33us) client_negotiation.cc:240] Negotiation successful
0926 06:21:58.990178 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1594,"thread_start_us":81,"threads_started":1}
I20260926 06:21:58.990899 16676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:58.971750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41120 (local address 127.4.202.129:37227)
0926 06:21:58.984297 (+ 12547us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:58.984301 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:58.984311 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:21:58.984444 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:58.984452 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:58.984480 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:58.985363 (+ 883us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:21:58.986935 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:58.987892 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:58.989222 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:58.989937 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:58.990680 (+ 743us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:58.990745 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 06:21:58.990771 (+ 26us) server_negotiation.cc:299] Negotiation successful
0926 06:21:58.990809 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12513,"thread_start_us":53,"threads_started":1}
I20260926 06:21:58.991616 16323 tablet_service.cc:1467] Processing CreateTablet for tablet e4236b92d1de45a9897b60ccc4edc377 (DEFAULT_TABLE table=test-table [id=7c9a39aa8f9c41ccad066ce490986717]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:21:58.991943 16323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4236b92d1de45a9897b60ccc4edc377. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:21:58.995110 16682 tablet_bootstrap.cc:492] T e4236b92d1de45a9897b60ccc4edc377 P 97d7edd439a74f5aa477f951e389cb8a: Bootstrap starting.
I20260926 06:21:58.995774 16682 tablet_bootstrap.cc:654] T e4236b92d1de45a9897b60ccc4edc377 P 97d7edd439a74f5aa477f951e389cb8a: Neither blocks nor log segments found. Creating new log.
I20260926 06:21:58.996136 16682 log.cc:824] T e4236b92d1de45a9897b60ccc4edc377 P 97d7edd439a74f5aa477f951e389cb8a: Log is configured to *not* fsync() on all Append() calls
I20260926 06:21:58.996929 16682 tablet_bootstrap.cc:492] T e4236b92d1de45a9897b60ccc4edc377 P 97d7edd439a74f5aa477f951e389cb8a: No bootstrap required, opened a new log
I20260926 06:21:58.996989 16682 ts_tablet_manager.cc:1397] T e4236b92d1de45a9897b60ccc4edc377 P 97d7edd439a74f5aa477f951e389cb8a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:21:58.997916 16680 tablet_bootstrap.cc:492] T e4236b92d1de45a9897b60ccc4edc377 P 4341265376174d3e96019a104fc3da7f: Bootstrap starting.
I20260926 06:21:58.998703 16681 tablet_bootstrap.cc:492] T e4236b92d1de45a9897b60ccc4edc377 P a1f030d3416040ffb9a09038988199e1: Bootstrap starting.
I20260926 06:21:58.999553 16681 tablet_bootstrap.cc:654] T e4236b92d1de45a9897b60ccc4edc377 P a1f030d3416040ffb9a09038988199e1: Neither blocks nor log segments found. Creating new log.
I20260926 06:21:58.999629 16682 raft_consensus.cc:348] T e4236b92d1de45a9897b60ccc4edc377 P 97d7edd439a74f5aa477f951e389cb8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4341265376174d3e96019a104fc3da7f" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 39919 } } peers { permanent_uuid: "97d7edd439a74f5aa477f951e389cb8a" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37227 } } peers { permanent_uuid: "a1f030d3416040ffb9a09038988199e1" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 42621 } }
I20260926 06:21:58.999789 16682 raft_consensus.cc:374] T e4236b92d1de45a9897b60ccc4edc377 P 97d7edd439a74f5aa477f951e389cb8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:21:58.999810 16681 log.cc:824] T e4236b92d1de45a9897b60ccc4edc377 P a1f030d3416040ffb9a09038988199e1: Log is configured to *not* fsync() on all Append() calls
I20260926 06:21:58.999821 16682 raft_consensus.cc:729] T e4236b92d1de45a9897b60ccc4edc377 P 97d7edd439a74f5aa477f951e389cb8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97d7edd439a74f5aa477f951e389cb8a, State: Initialized, Role: FOLLOWER
I20260926 06:21:59.000195 16680 tablet_bootstrap.cc:654] T e4236b92d1de45a9897b60ccc4edc377 P 4341265376174d3e96019a104fc3da7f: Neither blocks nor log segments found. Creating new log.
I20260926 06:21:59.000244 16682 consensus_queue.cc:260] T e4236b92d1de45a9897b60ccc4edc377 P 97d7edd439a74f5aa477f951e389cb8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4341265376174d3e96019a104fc3da7f" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 39919 } } peers { permanent_uuid: "97d7edd439a74f5aa477f951e389cb8a" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37227 } } peers { permanent_uuid: "a1f030d3416040ffb9a09038988199e1" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 42621 } }
I20260926 06:21:59.000567 16680 log.cc:824] T e4236b92d1de45a9897b60ccc4edc377 P 4341265376174d3e96019a104fc3da7f: Log is configured to *not* fsync() on all Append() calls
I20260926 06:21:59.000628 16682 ts_tablet_manager.cc:1428] T e4236b92d1de45a9897b60ccc4edc377 P 97d7edd439a74f5aa477f951e389cb8a: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260926 06:21:59.000778 16389 heartbeater.cc:503] Master 127.4.202.190:39563 was elected leader, sending a full tablet report...
I20260926 06:21:59.002954 16680 tablet_bootstrap.cc:492] T e4236b92d1de45a9897b60ccc4edc377 P 4341265376174d3e96019a104fc3da7f: No bootstrap required, opened a new log
I20260926 06:21:59.003023 16680 ts_tablet_manager.cc:1397] T e4236b92d1de45a9897b60ccc4edc377 P 4341265376174d3e96019a104fc3da7f: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260926 06:21:59.005215 16680 raft_consensus.cc:348] T e4236b92d1de45a9897b60ccc4edc377 P 4341265376174d3e96019a104fc3da7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4341265376174d3e96019a104fc3da7f" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 39919 } } peers { permanent_uuid: "97d7edd439a74f5aa477f951e389cb8a" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37227 } } peers { permanent_uuid: "a1f030d3416040ffb9a09038988199e1" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 42621 } }
I20260926 06:21:59.005363 16680 raft_consensus.cc:374] T e4236b92d1de45a9897b60ccc4edc377 P 4341265376174d3e96019a104fc3da7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:21:59.005406 16680 raft_consensus.cc:729] T e4236b92d1de45a9897b60ccc4edc377 P 4341265376174d3e96019a104fc3da7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4341265376174d3e96019a104fc3da7f, State: Initialized, Role: FOLLOWER
I20260926 06:21:59.005522 16680 consensus_queue.cc:260] T e4236b92d1de45a9897b60ccc4edc377 P 4341265376174d3e96019a104fc3da7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4341265376174d3e96019a104fc3da7f" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 39919 } } peers { permanent_uuid: "97d7edd439a74f5aa477f951e389cb8a" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37227 } } peers { permanent_uuid: "a1f030d3416040ffb9a09038988199e1" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 42621 } }
I20260926 06:21:59.005751 16680 ts_tablet_manager.cc:1428] T e4236b92d1de45a9897b60ccc4edc377 P 4341265376174d3e96019a104fc3da7f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 06:21:59.005895 16527 heartbeater.cc:503] Master 127.4.202.190:39563 was elected leader, sending a full tablet report...
I20260926 06:21:59.006381 16681 tablet_bootstrap.cc:492] T e4236b92d1de45a9897b60ccc4edc377 P a1f030d3416040ffb9a09038988199e1: No bootstrap required, opened a new log
I20260926 06:21:59.006446 16681 ts_tablet_manager.cc:1397] T e4236b92d1de45a9897b60ccc4edc377 P a1f030d3416040ffb9a09038988199e1: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260926 06:21:59.007884 16681 raft_consensus.cc:348] T e4236b92d1de45a9897b60ccc4edc377 P a1f030d3416040ffb9a09038988199e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4341265376174d3e96019a104fc3da7f" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 39919 } } peers { permanent_uuid: "97d7edd439a74f5aa477f951e389cb8a" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37227 } } peers { permanent_uuid: "a1f030d3416040ffb9a09038988199e1" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 42621 } }
I20260926 06:21:59.008021 16681 raft_consensus.cc:374] T e4236b92d1de45a9897b60ccc4edc377 P a1f030d3416040ffb9a09038988199e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:21:59.008091 16681 raft_consensus.cc:729] T e4236b92d1de45a9897b60ccc4edc377 P a1f030d3416040ffb9a09038988199e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1f030d3416040ffb9a09038988199e1, State: Initialized, Role: FOLLOWER
I20260926 06:21:59.008200 16681 consensus_queue.cc:260] T e4236b92d1de45a9897b60ccc4edc377 P a1f030d3416040ffb9a09038988199e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4341265376174d3e96019a104fc3da7f" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 39919 } } peers { permanent_uuid: "97d7edd439a74f5aa477f951e389cb8a" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37227 } } peers { permanent_uuid: "a1f030d3416040ffb9a09038988199e1" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 42621 } }
I20260926 06:21:59.008474 16661 heartbeater.cc:503] Master 127.4.202.190:39563 was elected leader, sending a full tablet report...
I20260926 06:21:59.008463 16681 ts_tablet_manager.cc:1428] T e4236b92d1de45a9897b60ccc4edc377 P a1f030d3416040ffb9a09038988199e1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
Sep 26 06:21:59 dist-test-slave-pd0t krb5kdc[15701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403718, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for HTTP/127.4.202.148@KRBTEST.COM
I20260926 06:21:59.112490 16685 raft_consensus.cc:482] T e4236b92d1de45a9897b60ccc4edc377 P 97d7edd439a74f5aa477f951e389cb8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:21:59.112650 16685 raft_consensus.cc:504] T e4236b92d1de45a9897b60ccc4edc377 P 97d7edd439a74f5aa477f951e389cb8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4341265376174d3e96019a104fc3da7f" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 39919 } } peers { permanent_uuid: "97d7edd439a74f5aa477f951e389cb8a" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37227 } } peers { permanent_uuid: "a1f030d3416040ffb9a09038988199e1" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 42621 } }
I20260926 06:21:59.113217 16685 leader_election.cc:288] T e4236b92d1de45a9897b60ccc4edc377 P 97d7edd439a74f5aa477f951e389cb8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4341265376174d3e96019a104fc3da7f (127.4.202.130:39919), a1f030d3416040ffb9a09038988199e1 (127.4.202.131:42621)
I20260926 06:21:59.117372 16678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:59.113808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:48377 (local address 127.4.202.130:39919)
0926 06:21:59.113911 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:59.113913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:59.113922 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:21:59.113951 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:59.113952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:59.113961 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:59.114026 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:21:59.114245 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:59.114998 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:59.116465 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:59.117017 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:59.117063 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:59.117072 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 06:21:59.117267 (+ 195us) server_negotiation.cc:299] Negotiation successful
0926 06:21:59.117302 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 06:21:59.117643 16689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:59.113444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:39919 (local address 127.4.202.129:48377)
0926 06:21:59.113829 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:59.113839 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:59.113886 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:59.114084 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:59.114085 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:59.114088 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:21:59.114172 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:59.114174 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:59.115116 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:59.115117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:59.116359 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:59.116362 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:59.117127 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:59.117128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:59.117197 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:59.117200 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:21:59.117488 (+ 288us) client_negotiation.cc:240] Negotiation successful
0926 06:21:59.117528 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":104,"threads_started":1}
I20260926 06:21:59.118181 16478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4236b92d1de45a9897b60ccc4edc377" candidate_uuid: "97d7edd439a74f5aa477f951e389cb8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4341265376174d3e96019a104fc3da7f" is_pre_election: true
I20260926 06:21:59.118340 16478 raft_consensus.cc:2454] T e4236b92d1de45a9897b60ccc4edc377 P 4341265376174d3e96019a104fc3da7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97d7edd439a74f5aa477f951e389cb8a in term 0.
I20260926 06:21:59.118606 16279 leader_election.cc:302] T e4236b92d1de45a9897b60ccc4edc377 P 97d7edd439a74f5aa477f951e389cb8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4341265376174d3e96019a104fc3da7f, 97d7edd439a74f5aa477f951e389cb8a; no voters:
I20260926 06:21:59.118716 16685 raft_consensus.cc:2781] T e4236b92d1de45a9897b60ccc4edc377 P 97d7edd439a74f5aa477f951e389cb8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:21:59.118772 16685 raft_consensus.cc:482] T e4236b92d1de45a9897b60ccc4edc377 P 97d7edd439a74f5aa477f951e389cb8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:21:59.118793 16685 raft_consensus.cc:3037] T e4236b92d1de45a9897b60ccc4edc377 P 97d7edd439a74f5aa477f951e389cb8a [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:21:59.119618 16685 raft_consensus.cc:504] T e4236b92d1de45a9897b60ccc4edc377 P 97d7edd439a74f5aa477f951e389cb8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4341265376174d3e96019a104fc3da7f" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 39919 } } peers { permanent_uuid: "97d7edd439a74f5aa477f951e389cb8a" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37227 } } peers { permanent_uuid: "a1f030d3416040ffb9a09038988199e1" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 42621 } }
I20260926 06:21:59.119724 16685 leader_election.cc:288] T e4236b92d1de45a9897b60ccc4edc377 P 97d7edd439a74f5aa477f951e389cb8a [CANDIDATE]: Term 1 election: Requested vote from peers 4341265376174d3e96019a104fc3da7f (127.4.202.130:39919), a1f030d3416040ffb9a09038988199e1 (127.4.202.131:42621)
I20260926 06:21:59.120000 16478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4236b92d1de45a9897b60ccc4edc377" candidate_uuid: "97d7edd439a74f5aa477f951e389cb8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4341265376174d3e96019a104fc3da7f"
I20260926 06:21:59.120069 16478 raft_consensus.cc:3037] T e4236b92d1de45a9897b60ccc4edc377 P 4341265376174d3e96019a104fc3da7f [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:21:59.121321 16478 raft_consensus.cc:2454] T e4236b92d1de45a9897b60ccc4edc377 P 4341265376174d3e96019a104fc3da7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97d7edd439a74f5aa477f951e389cb8a in term 1.
I20260926 06:21:59.121330 16679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:59.114033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:44037 (local address 127.4.202.131:42621)
0926 06:21:59.114131 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:59.114132 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:59.118128 (+ 3996us) server_negotiation.cc:407] Connection header received
0926 06:21:59.118228 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:59.118229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:59.118241 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:59.118272 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:21:59.118420 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:59.119181 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:59.120432 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:59.120988 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:59.121033 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:59.121041 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:21:59.121226 (+ 185us) server_negotiation.cc:299] Negotiation successful
0926 06:21:59.121264 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 06:21:59.121531 16689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:59.113639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:42621 (local address 127.4.202.129:44037)
0926 06:21:59.118081 (+ 4442us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:59.118091 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:21:59.118154 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:21:59.118328 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:21:59.118329 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:21:59.118330 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:21:59.118356 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:59.118357 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:59.119292 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:59.119292 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:59.120314 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:21:59.120316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:21:59.121095 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:21:59.121096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:21:59.121155 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:59.121157 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:21:59.121433 (+ 276us) client_negotiation.cc:240] Negotiation successful
0926 06:21:59.121469 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4431,"thread_start_us":52,"threads_started":1}
I20260926 06:21:59.121698 16279 leader_election.cc:302] T e4236b92d1de45a9897b60ccc4edc377 P 97d7edd439a74f5aa477f951e389cb8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4341265376174d3e96019a104fc3da7f, 97d7edd439a74f5aa477f951e389cb8a; no voters:
I20260926 06:21:59.122010 16615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4236b92d1de45a9897b60ccc4edc377" candidate_uuid: "97d7edd439a74f5aa477f951e389cb8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1f030d3416040ffb9a09038988199e1" is_pre_election: true
I20260926 06:21:59.122138 16615 raft_consensus.cc:2454] T e4236b92d1de45a9897b60ccc4edc377 P a1f030d3416040ffb9a09038988199e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97d7edd439a74f5aa477f951e389cb8a in term 0.
I20260926 06:21:59.122231 16614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4236b92d1de45a9897b60ccc4edc377" candidate_uuid: "97d7edd439a74f5aa477f951e389cb8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1f030d3416040ffb9a09038988199e1"
I20260926 06:21:59.122279 16614 raft_consensus.cc:3037] T e4236b92d1de45a9897b60ccc4edc377 P a1f030d3416040ffb9a09038988199e1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:21:59.123045 16614 raft_consensus.cc:2454] T e4236b92d1de45a9897b60ccc4edc377 P a1f030d3416040ffb9a09038988199e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97d7edd439a74f5aa477f951e389cb8a in term 1.
I20260926 06:21:59.123535 16685 raft_consensus.cc:2781] T e4236b92d1de45a9897b60ccc4edc377 P 97d7edd439a74f5aa477f951e389cb8a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:21:59.123679 16685 raft_consensus.cc:686] T e4236b92d1de45a9897b60ccc4edc377 P 97d7edd439a74f5aa477f951e389cb8a [term 1 LEADER]: Becoming Leader. State: Replica: 97d7edd439a74f5aa477f951e389cb8a, State: Running, Role: LEADER
I20260926 06:21:59.123756 16685 consensus_queue.cc:237] T e4236b92d1de45a9897b60ccc4edc377 P 97d7edd439a74f5aa477f951e389cb8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4341265376174d3e96019a104fc3da7f" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 39919 } } peers { permanent_uuid: "97d7edd439a74f5aa477f951e389cb8a" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37227 } } peers { permanent_uuid: "a1f030d3416040ffb9a09038988199e1" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 42621 } }
I20260926 06:21:59.125051 16177 catalog_manager.cc:5495] T e4236b92d1de45a9897b60ccc4edc377 P 97d7edd439a74f5aa477f951e389cb8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 97d7edd439a74f5aa477f951e389cb8a (127.4.202.129). New cstate: current_term: 1 leader_uuid: "97d7edd439a74f5aa477f951e389cb8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4341265376174d3e96019a104fc3da7f" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 39919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97d7edd439a74f5aa477f951e389cb8a" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1f030d3416040ffb9a09038988199e1" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 42621 } health_report { overall_health: UNKNOWN } } }
I20260926 06:21:59.156152 16676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:59.152898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41136 (local address 127.4.202.129:37227)
0926 06:21:59.152951 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0926 06:21:59.152953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:21:59.152968 (+ 15us) server_negotiation.cc:407] Connection header received
0926 06:21:59.153075 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:21:59.153076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:21:59.153096 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:21:59.153153 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:21:59.153324 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:59.153944 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:59.155063 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:21:59.155344 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:21:59.155411 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:21:59.155615 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:21:59.155749 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:21:59.155819 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 06:21:59.156053 (+ 234us) server_negotiation.cc:299] Negotiation successful
0926 06:21:59.156075 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 06:21:59.158471 16614 raft_consensus.cc:1261] T e4236b92d1de45a9897b60ccc4edc377 P a1f030d3416040ffb9a09038988199e1 [term 1 FOLLOWER]: Refusing update from remote peer 97d7edd439a74f5aa477f951e389cb8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:21:59.158910 16685 consensus_queue.cc:1035] T e4236b92d1de45a9897b60ccc4edc377 P 97d7edd439a74f5aa477f951e389cb8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1f030d3416040ffb9a09038988199e1" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 42621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:21:59.160683 16478 raft_consensus.cc:1261] T e4236b92d1de45a9897b60ccc4edc377 P 4341265376174d3e96019a104fc3da7f [term 1 FOLLOWER]: Refusing update from remote peer 97d7edd439a74f5aa477f951e389cb8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:21:59.161125 16685 consensus_queue.cc:1035] T e4236b92d1de45a9897b60ccc4edc377 P 97d7edd439a74f5aa477f951e389cb8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4341265376174d3e96019a104fc3da7f" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 39919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:21:59.165222 16694 mvcc.cc:205] Tried to move back new op lower bound from 7333493633670053888 to 7333493633531494400. Current Snapshot: MvccSnapshot[applied={T|T < 7333493633670053888}]
I20260926 06:21:59.165446 16697 mvcc.cc:205] Tried to move back new op lower bound from 7333493633670053888 to 7333493633531494400. Current Snapshot: MvccSnapshot[applied={T|T < 7333493633670053888}]
I20260926 06:21:59.167426 16695 mvcc.cc:205] Tried to move back new op lower bound from 7333493633670053888 to 7333493633531494400. Current Snapshot: MvccSnapshot[applied={T|T < 7333493633670053888}]
I20260926 06:21:59.170115 16177 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:21:59.170200 16177 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:21:59.170970 16177 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 42059e997cd347c58681fb95bd2bc314: Sending DeleteTablet for 3 replicas of tablet e4236b92d1de45a9897b60ccc4edc377
I20260926 06:21:59.171288 16460 tablet_service.cc:1514] Processing DeleteTablet for tablet e4236b92d1de45a9897b60ccc4edc377 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:21:59 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:47454
I20260926 06:21:59.171337 16580 tablet_service.cc:1514] Processing DeleteTablet for tablet e4236b92d1de45a9897b60ccc4edc377 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:21:59 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:39182
I20260926 06:21:59.171763 16323 tablet_service.cc:1514] Processing DeleteTablet for tablet e4236b92d1de45a9897b60ccc4edc377 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:21:59 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:41120
I20260926 06:21:59.172231 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 16258
I20260926 06:21:59.173913 16704 tablet_replica.cc:330] T e4236b92d1de45a9897b60ccc4edc377 P a1f030d3416040ffb9a09038988199e1: stopping tablet replica
I20260926 06:21:59.174094 16704 raft_consensus.cc:2229] T e4236b92d1de45a9897b60ccc4edc377 P a1f030d3416040ffb9a09038988199e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:21:59.174244 16704 raft_consensus.cc:2258] T e4236b92d1de45a9897b60ccc4edc377 P a1f030d3416040ffb9a09038988199e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:21:59.175261 16704 ts_tablet_manager.cc:1905] T e4236b92d1de45a9897b60ccc4edc377 P a1f030d3416040ffb9a09038988199e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:21:59.179107 16704 ts_tablet_manager.cc:1918] T e4236b92d1de45a9897b60ccc4edc377 P a1f030d3416040ffb9a09038988199e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:21:59.179230 16704 log.cc:1196] T e4236b92d1de45a9897b60ccc4edc377 P a1f030d3416040ffb9a09038988199e1: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790403591640726-4906-0/minicluster-data/ts-2/wal/wals/e4236b92d1de45a9897b60ccc4edc377
I20260926 06:21:59.179855 16704 ts_tablet_manager.cc:1939] T e4236b92d1de45a9897b60ccc4edc377 P a1f030d3416040ffb9a09038988199e1: Deleting consensus metadata
I20260926 06:21:59.180328 16164 catalog_manager.cc:4841] TS a1f030d3416040ffb9a09038988199e1 (127.4.202.131:42621): tablet e4236b92d1de45a9897b60ccc4edc377 (table test-table [id=7c9a39aa8f9c41ccad066ce490986717]) successfully deleted
W20260926 06:21:59.182628 16164 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.202.129:37227 (error 108)
I20260926 06:21:59.183476 16674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:59.183349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:37227 (local address 127.0.0.1:41146)
0926 06:21:59.183402 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:59.183442 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.129:37227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260926 06:21:59.183475 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 16395
W20260926 06:21:59.183862 16164 catalog_manager.cc:4584] TS 97d7edd439a74f5aa477f951e389cb8a (127.4.202.129:37227): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e4236b92d1de45a9897b60ccc4edc377: Network error: Client connection negotiation failed: client connection to 127.4.202.129:37227: connect: Connection refused (error 111)
I20260926 06:21:59.189440 16674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:21:59.189333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:39919 (local address 127.0.0.1:47456)
0926 06:21:59.189375 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0926 06:21:59.189399 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.130:39919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260926 06:21:59.189668 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 16532
I20260926 06:21:59.196246 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 16145
I20260926 06:21:59.204564 4906 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260926 06:21:59.538475 4906 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260926 06:21:59.538561 4906 mini_ranger.cc:66] Stopping Ranger...
I20260926 06:21:59.882534 4906 mini_ranger.cc:68] Stopped Ranger
2026-09-26 06:21:59.882 UTC [15731] LOG: received smart shutdown request
2026-09-26 06:21:59.885 UTC [15731] LOG: background worker "logical replication launcher" (PID 15740) exited with exit code 1
2026-09-26 06:21:59.886 UTC [15735] LOG: shutting down
2026-09-26 06:22:00.039 UTC [15731] LOG: database system is shut down
2026-09-26T06:22:00Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (43445 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:22:00 dist-test-slave-pd0t krb5kdc[16715](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:22:00 dist-test-slave-pd0t krb5kdc[16715](info): set up 2 sockets
Sep 26 06:22:00 dist-test-slave-pd0t krb5kdc[16715](info): commencing operation
krb5kdc: starting...
W20260926 06:22:01.166643 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:22:01 dist-test-slave-pd0t krb5kdc[16715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:22:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:22:01Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.4.202.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.4.202.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.4.202.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/rangeradmin_127.4.202.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.4.202.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/rangeradmin_127.4.202.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.4.202.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.4.202.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.4.202.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/rangerlookup_127.4.202.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.4.202.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/rangerlookup_127.4.202.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.202.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.4.202.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.202.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/HTTP_127.4.202.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.202.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/HTTP_127.4.202.148@KRBTEST.COM.keytab.
I20260926 06:22:01.317404 4906 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskm7eEJS/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/postgres -l logfile start
2026-09-26 06:22:03.753 UTC [16745] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-26 06:22:03.754 UTC [16745] LOG: listening on IPv4 address "127.4.202.148", port 55107
2026-09-26 06:22:03.757 UTC [16745] LOG: listening on Unix socket "/tmp/.s.PGSQL.55107"
2026-09-26 06:22:03.792 UTC [16748] LOG: database system was shut down at 2026-09-26 06:22:02 UTC
2026-09-26 06:22:03.796 UTC [16745] LOG: database system is ready to accept connections
I20260926 06:22:04.726267 4906 mini_postgres.cc:96] Postgres bound to 55107
2026-09-26 06:22:04.731 UTC [16756] FATAL: database "slave" does not exist
127.4.202.148:55107 - accepting connections
I20260926 06:22:04.732470 4906 mini_ranger.cc:161] Starting Ranger...
I20260926 06:22:04.746663 4906 mini_ranger.cc:84] Created miniranger Postgres user
I20260926 06:22:05.221529 4906 mini_ranger.cc:87] Created ranger Postgres database
I20260926 06:22:05.221596 4906 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ranger-admin
2026-09-26 06:22:05,509 [I] DB FLAVOR :POSTGRES
2026-09-26 06:22:05,509 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 06:22:05,510 [I] Checking connection..
2026-09-26 06:22:05,510 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:55107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-26 06:22:05,757 [I] Checking connection passed.
2026-09-26 06:22:05,757 [I] --------- Verifying version history table ---------
2026-09-26 06:22:05,758 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:55107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 06:22:05,991 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 06:22:05,991 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:55107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 06:22:06,220 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 06:22:06,220 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-26 06:22:06,221 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:55107/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-26 06:22:06.439 UTC [16820] WARNING: there is no transaction in progress
2026-09-26 06:22:06,445 [I] create_dbversion_catalog.sql file imported successfully
2026-09-26 06:22:06,445 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:55107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 06:22:06,682 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-26 06:22:06,683 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 06:22:06,683 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:55107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-26 06:22:06,901 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:55107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-26 06:22:07,146 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:55107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-pd0t.c.gcp-upstream.internal','N') ;"
2026-09-26 06:22:07,385 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:55107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 06:22:07,639 [I] Table x_portal_user does not exist in database ranger
2026-09-26 06:22:07,639 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:55107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 06:22:07,883 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-26 06:22:07,884 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:55107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 06:22:08,115 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:55107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 06:22:08,362 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-26 06:22:08,363 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:55107/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-26 06:22:09.291 UTC [16928] WARNING: there is no transaction in progress
2026-09-26 06:22:09.307 UTC [16928] WARNING: there is no transaction in progress
2026-09-26 06:22:09.324 UTC [16928] WARNING: there is no transaction in progress
2026-09-26 06:22:09.339 UTC [16928] WARNING: there is no transaction in progress
2026-09-26 06:22:09.357 UTC [16928] WARNING: there is no transaction in progress
2026-09-26 06:22:09.376 UTC [16928] WARNING: there is no transaction in progress
2026-09-26 06:22:09.437 UTC [16928] WARNING: there is no transaction in progress
2026-09-26 06:22:09.446 UTC [16928] WARNING: there is no transaction in progress
2026-09-26 06:22:09.463 UTC [16928] WARNING: there is no transaction in progress
2026-09-26 06:22:09.476 UTC [16928] WARNING: there is no transaction in progress
2026-09-26 06:22:09.487 UTC [16928] WARNING: there is no transaction in progress
2026-09-26 06:22:09.500 UTC [16928] WARNING: there is no transaction in progress
2026-09-26 06:22:09.509 UTC [16928] WARNING: there is no transaction in progress
2026-09-26 06:22:09.515 UTC [16928] WARNING: there is no transaction in progress
2026-09-26 06:22:09.522 UTC [16928] WARNING: there is no transaction in progress
2026-09-26 06:22:09.528 UTC [16928] WARNING: there is no transaction in progress
2026-09-26 06:22:10,143 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-26 06:22:10,143 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:55107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-26 06:22:10,357 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-26 06:22:10,358 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:55107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 06:22:10,583 [I] Table x_portal_user already exists in database 'ranger'
2026-09-26 06:22:10,583 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:55107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 06:22:10,801 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-26 06:22:10,801 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:55107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 06:22:11,026 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:55107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 06:22:11,278 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:55107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-pd0t.c.gcp-upstream.internal';"
2026-09-26 06:22:11,506 [I] CORE_DB_SCHEMA import status has been updated
2026-09-26 06:22:11,507 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:55107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-26 06:22:11,743 [I] DB_PATCHES have already been applied
I20260926 06:22:11.748369 4906 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ranger-admin:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskm7eEJS/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260926 06:22:11.748430 4906 mini_ranger.cc:193] Using host: 127.4.202.148
I20260926 06:22:11.751246 4906 mini_ranger.cc:227] Ranger admin URL: http://127.4.202.148:50601
Sep 26, 2026 6:22:12 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 6:22:12 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 6:22:12 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ranger-admin/ews/webapp
Sep 26, 2026 6:22:12 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 26, 2026 6:22:12 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 26, 2026 6:22:12 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ranger-admin/ews/webapp].
Sep 26, 2026 6:22:12 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260926 06:22:12.851039 4906 mini_ranger.cc:160] Time spent starting Ranger: real 8.119s user 0.003s sys 0.005s
Sep 26, 2026 6:22:31 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 26, 2026 6:22:31 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 26, 2026 6:22:31 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 26, 2026 6:22:31 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 26, 2026 6:22:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 26, 2026 6:22:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 26, 2026 6:22:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 26, 2026 6:22:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 26, 2026 6:22:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 26, 2026 6:22:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 26, 2026 6:22:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 26, 2026 6:22:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 26, 2026 6:22:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 26, 2026 6:22:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 26, 2026 6:22:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 26, 2026 6:22:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 26, 2026 6:22:31 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 26, 2026 6:22:32 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260926 06:22:32.767134 4906 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.4.202.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.4.202.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.4.202.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/rangerkms_127.4.202.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.4.202.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/rangerkms_127.4.202.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.202.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.4.202.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.202.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/HTTP_127.4.202.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.202.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/HTTP_127.4.202.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 26 06:22:32 dist-test-slave-pd0t krb5kdc[16715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403752, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260926 06:22:32.845284 4906 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260926 06:22:32.853977 4906 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260926 06:22:33.683801 4906 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260926 06:22:33.683882 4906 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ranger-kms
I20260926 06:22:33.683900 4906 mini_ranger_kms.cc:227] Using postgres at 127.4.202.148:55107
2026-09-26 06:22:33,845 [I] DB FLAVOR :POSTGRES
2026-09-26 06:22:33,846 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 06:22:33,846 [I] Checking connection
2026-09-26 06:22:33,846 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:55107/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-26 06:22:34,076 [I] connection success
2026-09-26 06:22:34,077 [I] --------- Verifying Ranger DB tables ---------
2026-09-26 06:22:34,077 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-26 06:22:34,077 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:55107/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-26 06:22:34,296 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-26 06:22:34,297 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 06:22:34,297 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-26 06:22:34,297 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.202.148:55107/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-26 06:22:34,534 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260926 06:22:34.961498 4906 mini_ranger_kms.cc:310] Created kms service
I20260926 06:22:35.132128 4906 mini_ranger_kms.cc:326] Created kudu user
I20260926 06:22:35.171679 4906 mini_ranger_kms.cc:343] Created rangerkms user
Sep 26 06:22:35 dist-test-slave-pd0t krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403752, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.4.202.148@KRBTEST.COM
I20260926 06:22:35.541757 4906 mini_ranger_kms.cc:384] Added ranger policy
I20260926 06:22:35.541893 4906 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ranger-kms:/tmp/dist-test-taskm7eEJS/build/release/bin/postgresql.jar:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskm7eEJS/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260926 06:22:35.541921 4906 mini_ranger_kms.cc:240] Using host: 127.4.202.148
I20260926 06:22:35.545357 4906 mini_ranger_kms.cc:276] Ranger KMS PID: 17108
I20260926 06:22:35.545439 4906 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.4.202.148:36023
Sep 26, 2026 6:22:36 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 6:22:36 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 6:22:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 26, 2026 6:22:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 26, 2026 6:22:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ranger-kms/ews/webapp].
Sep 26, 2026 6:22:37 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260926 06:22:37.780918 4906 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.936s user 0.001s sys 0.144s
I20260926 06:22:37.781015 4906 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260926 06:22:37.781071 4906 mini_ranger_kms.cc:401] 127.4.202.148:36023/kms/v1/keys
Sep 26 06:22:37 dist-test-slave-pd0t krb5kdc[16715](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790403757, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.202.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 26 06:22:38 dist-test-slave-pd0t krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403757, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.202.148@KRBTEST.COM for HTTP/127.4.202.148@KRBTEST.COM
Sep 26 06:22:38 dist-test-slave-pd0t krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403757, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.202.148@KRBTEST.COM for HTTP/127.4.202.148@KRBTEST.COM
Sep 26 06:22:40 dist-test-slave-pd0t krb5kdc[16715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.4.202.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
I20260926 06:22:40.825503 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:43269
--webserver_interface=127.4.202.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:41935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:43269
--ranger_config_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.202.148:36023/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:22:40.908254 17159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:22:40.908475 17159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:22:40.908511 17159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:22:40.909601 17159 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:22:40.909662 17159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:22:40.909685 17159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:22:40.909713 17159 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:22:40.909734 17159 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:22:40.911401 17159 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:41935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/master-0/wal
--ranger_kms_url=127.4.202.148:36023/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:43269
--ranger_config_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:43269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:22:40.911667 17159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:22:40.911878 17159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:22:40.916894 17164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:22:40.916975 17159 server_base.cc:1034] running on GCE node
W20260926 06:22:40.916883 17167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:40.916868 17165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:22:40.917353 17159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:22:40.917588 17159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:22:40.918808 17159 hybrid_clock.cc:648] HybridClock initialized: now 1790403760918791 us; error 36 us; skew 500 ppm
Sep 26 06:22:40 dist-test-slave-pd0t krb5kdc[16715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403760, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:22:40.921191 17159 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:22:40.921533 17159 webserver.cc:460] Webserver started at http://127.4.202.190:37483/ using document root <none> and password file <none>
I20260926 06:22:40.921690 17159 fs_manager.cc:359] Metadata directory not provided
I20260926 06:22:40.921723 17159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:22:40.921805 17159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 06:22:40 dist-test-slave-pd0t krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403760, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for HTTP/127.4.202.148@KRBTEST.COM
I20260926 06:22:40.973521 17159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "eba7d6e554794125855840f041ebba50"
format_stamp: "Formatted at 2026-09-26 06:22:40 on dist-test-slave-pd0t"
server_key: "45576aa682303c299a9c10a69762b962"
server_key_iv: "65a1eb7eb35c9ae5a30f3d592dc59eac"
server_key_version: "kuduclusterkey@0"
I20260926 06:22:40.973968 17159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "eba7d6e554794125855840f041ebba50"
format_stamp: "Formatted at 2026-09-26 06:22:40 on dist-test-slave-pd0t"
server_key: "45576aa682303c299a9c10a69762b962"
server_key_iv: "65a1eb7eb35c9ae5a30f3d592dc59eac"
server_key_version: "kuduclusterkey@0"
I20260926 06:22:40.975848 17159 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260926 06:22:40.989909 17174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:40.990130 17159 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 06:22:40.990234 17159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "eba7d6e554794125855840f041ebba50"
format_stamp: "Formatted at 2026-09-26 06:22:40 on dist-test-slave-pd0t"
server_key: "45576aa682303c299a9c10a69762b962"
server_key_iv: "65a1eb7eb35c9ae5a30f3d592dc59eac"
server_key_version: "kuduclusterkey@0"
I20260926 06:22:40.990321 17159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:22:41.021414 17159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:22:41.023664 17159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:22:41.023825 17159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:22:41.028856 17159 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:43269
I20260926 06:22:41.028916 17226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:43269 every 8 connection(s)
I20260926 06:22:41.029340 17159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:22:41.030035 17227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:22:41.031602 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 17159
I20260926 06:22:41.031705 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:22:41.031885 4906 external_mini_cluster.cc:1428] Setting key 6f7d408ca81a1603b0b63a8cbd489348
I20260926 06:22:41.032778 17227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50: Bootstrap starting.
I20260926 06:22:41.033504 17227 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50: Neither blocks nor log segments found. Creating new log.
I20260926 06:22:41.033824 17227 log.cc:824] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50: Log is configured to *not* fsync() on all Append() calls
I20260926 06:22:41.034574 17227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50: No bootstrap required, opened a new log
I20260926 06:22:41.036998 17227 raft_consensus.cc:348] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba7d6e554794125855840f041ebba50" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43269 } }
I20260926 06:22:41.037168 17227 raft_consensus.cc:374] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:22:41.037204 17227 raft_consensus.cc:729] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eba7d6e554794125855840f041ebba50, State: Initialized, Role: FOLLOWER
I20260926 06:22:41.037381 17227 consensus_queue.cc:260] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba7d6e554794125855840f041ebba50" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43269 } }
I20260926 06:22:41.037463 17227 raft_consensus.cc:388] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:22:41.037494 17227 raft_consensus.cc:482] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:22:41.037559 17227 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:22:41.038190 17227 raft_consensus.cc:504] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba7d6e554794125855840f041ebba50" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43269 } }
I20260926 06:22:41.038300 17227 leader_election.cc:302] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eba7d6e554794125855840f041ebba50; no voters:
Sep 26 06:22:41 dist-test-slave-pd0t krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:22:41.038498 17227 leader_election.cc:288] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:22:41.038552 17232 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:22:41.038758 17232 raft_consensus.cc:686] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [term 1 LEADER]: Becoming Leader. State: Replica: eba7d6e554794125855840f041ebba50, State: Running, Role: LEADER
I20260926 06:22:41.038869 17227 sys_catalog.cc:564] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:22:41.038903 17232 consensus_queue.cc:237] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba7d6e554794125855840f041ebba50" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43269 } }
I20260926 06:22:41.039227 17232 sys_catalog.cc:455] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eba7d6e554794125855840f041ebba50. Latest consensus state: current_term: 1 leader_uuid: "eba7d6e554794125855840f041ebba50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba7d6e554794125855840f041ebba50" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43269 } } }
I20260926 06:22:41.039296 17232 sys_catalog.cc:458] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [sys.catalog]: This master's current role is: LEADER
I20260926 06:22:41.039237 17234 sys_catalog.cc:455] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eba7d6e554794125855840f041ebba50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba7d6e554794125855840f041ebba50" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43269 } } }
I20260926 06:22:41.039590 17234 sys_catalog.cc:458] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [sys.catalog]: This master's current role is: LEADER
I20260926 06:22:41.039868 17236 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:22:41.041230 17236 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:22:41.041592 17227 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 06:22:41.042740 17230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:41.032430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50952 (local address 127.4.202.190:43269)
0926 06:22:41.032597 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:41.032606 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:41.032637 (+ 31us) server_negotiation.cc:407] Connection header received
0926 06:22:41.033200 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:41.033211 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:41.033271 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:41.033438 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:41.034194 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:41.035168 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:41.035905 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:41.036278 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:41.036399 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:41.038959 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:41.038963 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:41.038964 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:41.038975 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:41.040996 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:41.041413 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:41.041416 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:41.041416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:41.041500 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:41.041709 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:41.041710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:41.041710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:41.042014 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:41.042135 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:41.042408 (+ 273us) server_negotiation.cc:299] Negotiation successful
0926 06:22:41.042524 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":71,"threads_started":1}
I20260926 06:22:41.045725 17236 catalog_manager.cc:1347] Generated new cluster ID: 57e2cac7562f451bbb427d1367a66580
I20260926 06:22:41.045878 17236 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:22:41.072899 17236 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:22:41.073505 17236 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:22:41.076792 17236 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50: Generated new TSK 0
I20260926 06:22:41.076962 17236 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.202.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
I20260926 06:22:41.733722 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.202.148:36023/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.129:0
--local_ip_for_outbound_sockets=127.4.202.129
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:43269
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:41935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:22:41.841089 17272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:22:41.841303 17272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:22:41.841346 17272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:22:41.842530 17272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:22:41.842610 17272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:22:41.842662 17272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.129
I20260926 06:22:41.844079 17272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:41935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.4.202.148:36023/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:43269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.129
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:22:41.844326 17272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:22:41.844538 17272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:22:41.849577 17277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:41.849570 17278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:41.849570 17280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:22:41.850030 17272 server_base.cc:1034] running on GCE node
I20260926 06:22:41.850239 17272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:22:41.850476 17272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:22:41.851678 17272 hybrid_clock.cc:648] HybridClock initialized: now 1790403761851668 us; error 62 us; skew 500 ppm
Sep 26 06:22:41 dist-test-slave-pd0t krb5kdc[16715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403761, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:22:41.854734 17272 init.cc:377] Logged in from keytab as kudu/127.4.202.129@KRBTEST.COM (short username kudu)
I20260926 06:22:41.855280 17272 webserver.cc:460] Webserver started at http://127.4.202.129:38653/ using document root <none> and password file <none>
I20260926 06:22:41.855537 17272 fs_manager.cc:359] Metadata directory not provided
I20260926 06:22:41.855660 17272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:22:41.855831 17272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 06:22:41 dist-test-slave-pd0t krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403761, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for HTTP/127.4.202.148@KRBTEST.COM
I20260926 06:22:41.867859 17272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-0/data/instance:
uuid: "03a2050bb55341429a92ec2c53883730"
format_stamp: "Formatted at 2026-09-26 06:22:41 on dist-test-slave-pd0t"
server_key: "1b391592c078118fa82924f34794556c"
server_key_iv: "7ac6ce078c0cc8fb60636b632dc0c7e6"
server_key_version: "kuduclusterkey@0"
I20260926 06:22:41.868268 17272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance:
uuid: "03a2050bb55341429a92ec2c53883730"
format_stamp: "Formatted at 2026-09-26 06:22:41 on dist-test-slave-pd0t"
server_key: "1b391592c078118fa82924f34794556c"
server_key_iv: "7ac6ce078c0cc8fb60636b632dc0c7e6"
server_key_version: "kuduclusterkey@0"
I20260926 06:22:41.874929 17272 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.000s
I20260926 06:22:41.890761 17289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:41.890967 17272 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.001s
I20260926 06:22:41.891049 17272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-0/wal
uuid: "03a2050bb55341429a92ec2c53883730"
format_stamp: "Formatted at 2026-09-26 06:22:41 on dist-test-slave-pd0t"
server_key: "1b391592c078118fa82924f34794556c"
server_key_iv: "7ac6ce078c0cc8fb60636b632dc0c7e6"
server_key_version: "kuduclusterkey@0"
I20260926 06:22:41.891120 17272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:22:41.906509 17272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:22:41.908905 17272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:22:41.909077 17272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:22:41.909307 17272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:22:41.909682 17272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:22:41.909759 17272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:41.909845 17272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:22:41.909914 17272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:41.916916 17272 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.129:39293
I20260926 06:22:41.917028 17402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.129:39293 every 8 connection(s)
I20260926 06:22:41.917382 17272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
I20260926 06:22:41.921141 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 17272
I20260926 06:22:41.921231 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance
I20260926 06:22:41.921393 4906 external_mini_cluster.cc:1428] Setting key 31133fb8ea523ba582030ed96dbe7f46
Sep 26 06:22:41 dist-test-slave-pd0t krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403761, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:22:41.927750 17405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:41.918142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:43269 (local address 127.4.202.129:47251)
0926 06:22:41.918313 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:41.918360 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:41.918901 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:41.919262 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:41.919267 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:41.919448 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:22:41.919624 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:41.919628 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:41.920363 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:41.920364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:41.921398 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:41.921402 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:41.921933 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:41.921935 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:41.922048 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:41.922537 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:22:41.922545 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:22:41.924256 (+ 1711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:22:41.926019 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:41.926037 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:41.926040 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:41.926350 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:41.926634 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:41.926634 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:41.926635 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:41.926768 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:41.927074 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:22:41.927080 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:22:41.927236 (+ 156us) client_negotiation.cc:769] Sending connection context
0926 06:22:41.927327 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 06:22:41.927478 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":117,"threads_started":1}
I20260926 06:22:41.928222 17406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:41.918223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:47251 (local address 127.4.202.190:43269)
0926 06:22:41.919045 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:41.919047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:41.919054 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:22:41.919074 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:41.919074 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:41.919085 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:41.919188 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:41.919724 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:41.920250 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:41.921504 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:41.921790 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:41.921900 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:41.924368 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:41.924369 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:41.924370 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:41.924377 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:41.925915 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:41.926461 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:41.926462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:41.926462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:41.926547 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:41.926855 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:41.926856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:41.926856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:41.926993 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:41.927050 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:41.928021 (+ 971us) server_negotiation.cc:299] Negotiation successful
0926 06:22:41.928098 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":794,"thread_start_us":83,"threads_started":1}
I20260926 06:22:41.928745 17403 heartbeater.cc:347] Connected to a master server at 127.4.202.190:43269
I20260926 06:22:41.928884 17403 heartbeater.cc:464] Registering TS with master...
I20260926 06:22:41.929085 17403 heartbeater.cc:511] Master 127.4.202.190:43269 requested a full tablet report, sending...
I20260926 06:22:41.929684 17191 ts_manager.cc:194] Registered new tserver with Master: 03a2050bb55341429a92ec2c53883730 (127.4.202.129:39293)
I20260926 06:22:41.930658 17191 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.129@KRBTEST.COM'} at 127.4.202.129:47251
WARNING: no policy specified for kudu/127.4.202.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
I20260926 06:22:41.955536 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.202.148:36023/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.130:0
--local_ip_for_outbound_sockets=127.4.202.130
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:43269
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:41935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:22:42.047690 17409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:22:42.047900 17409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:22:42.047940 17409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:22:42.049067 17409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:22:42.049131 17409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:22:42.049176 17409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.130
Sep 26 06:22:42 dist-test-slave-pd0t krb5kdc[16715](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790403762, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:22:42.050943 17409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:41935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.4.202.148:36023/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:43269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.130
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:22:42.051169 17409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:22:42.051401 17409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:22:42.056828 17419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:42.056947 17416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:42.057180 17417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:22:42.057708 17409 server_base.cc:1034] running on GCE node
I20260926 06:22:42.057919 17409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:22:42.058182 17409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:22:42.059360 17409 hybrid_clock.cc:648] HybridClock initialized: now 1790403762059338 us; error 34 us; skew 500 ppm
Sep 26 06:22:42 dist-test-slave-pd0t krb5kdc[16715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403762, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:22:42.062542 17409 init.cc:377] Logged in from keytab as kudu/127.4.202.130@KRBTEST.COM (short username kudu)
I20260926 06:22:42.063053 17409 webserver.cc:460] Webserver started at http://127.4.202.130:33277/ using document root <none> and password file <none>
I20260926 06:22:42.063308 17409 fs_manager.cc:359] Metadata directory not provided
I20260926 06:22:42.063357 17409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:22:42.063473 17409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 06:22:42 dist-test-slave-pd0t krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403762, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for HTTP/127.4.202.148@KRBTEST.COM
I20260926 06:22:42.076673 17409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-1/data/instance:
uuid: "4b9f9bc12465427b9af617c47dda742e"
format_stamp: "Formatted at 2026-09-26 06:22:42 on dist-test-slave-pd0t"
server_key: "62d56f1a70376d5e56bdd7089b9f5114"
server_key_iv: "1ab27a5ad22a10c7435213e81041e5c6"
server_key_version: "kuduclusterkey@0"
I20260926 06:22:42.077170 17409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance:
uuid: "4b9f9bc12465427b9af617c47dda742e"
format_stamp: "Formatted at 2026-09-26 06:22:42 on dist-test-slave-pd0t"
server_key: "62d56f1a70376d5e56bdd7089b9f5114"
server_key_iv: "1ab27a5ad22a10c7435213e81041e5c6"
server_key_version: "kuduclusterkey@0"
I20260926 06:22:42.086476 17409 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.001s sys 0.000s
I20260926 06:22:42.102545 17426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:42.102860 17409 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260926 06:22:42.102988 17409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-1/wal
uuid: "4b9f9bc12465427b9af617c47dda742e"
format_stamp: "Formatted at 2026-09-26 06:22:42 on dist-test-slave-pd0t"
server_key: "62d56f1a70376d5e56bdd7089b9f5114"
server_key_iv: "1ab27a5ad22a10c7435213e81041e5c6"
server_key_version: "kuduclusterkey@0"
I20260926 06:22:42.103124 17409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:22:42.124333 17409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:22:42.126869 17409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:22:42.127002 17409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:22:42.127216 17409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:22:42.127548 17409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:22:42.127646 17409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:42.127728 17409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:22:42.127794 17409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:42.134095 17409 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.130:36317
I20260926 06:22:42.134537 17409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
Sep 26 06:22:42 dist-test-slave-pd0t krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403762, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:22:42.142769 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 17409
I20260926 06:22:42.142884 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance
I20260926 06:22:42.143054 4906 external_mini_cluster.cc:1428] Setting key 48ff45305a1d47747c97fd22b1b57b3e
I20260926 06:22:42.146199 17406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:42.135267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:40241 (local address 127.4.202.190:43269)
0926 06:22:42.135970 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:42.135972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:42.136037 (+ 65us) server_negotiation.cc:407] Connection header received
0926 06:22:42.136735 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:42.136736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:42.136751 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:42.136793 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:42.137581 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:42.138375 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:42.140068 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:42.140435 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:42.140484 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:42.142987 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:42.142988 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:42.142989 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:42.142996 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:42.144730 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:42.145196 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:42.145197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:42.145198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:42.145250 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:42.145489 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:42.145490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:42.145490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:42.145609 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:42.145650 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:42.146021 (+ 371us) server_negotiation.cc:299] Negotiation successful
0926 06:22:42.146106 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676}
I20260926 06:22:42.146700 17542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:42.135258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:43269 (local address 127.4.202.130:40241)
0926 06:22:42.135914 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:42.135961 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:42.136603 (+ 642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:42.136969 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:42.136974 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:42.137202 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:22:42.137455 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:42.137461 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:42.138480 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:42.138481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:42.139973 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:42.139976 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:42.140620 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:42.140621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:42.140757 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:42.141287 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:22:42.141296 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:22:42.142876 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:22:42.144835 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:42.144842 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:42.144844 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:42.145119 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:42.145315 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:42.145315 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:42.145316 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:42.145424 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:42.145697 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:22:42.145703 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:22:42.145873 (+ 170us) client_negotiation.cc:769] Sending connection context
0926 06:22:42.146283 (+ 410us) client_negotiation.cc:240] Negotiation successful
0926 06:22:42.146428 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":559,"threads_started":1}
I20260926 06:22:42.147030 17539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.130:36317 every 8 connection(s)
I20260926 06:22:42.154644 17540 heartbeater.cc:347] Connected to a master server at 127.4.202.190:43269
I20260926 06:22:42.154827 17540 heartbeater.cc:464] Registering TS with master...
I20260926 06:22:42.155120 17540 heartbeater.cc:511] Master 127.4.202.190:43269 requested a full tablet report, sending...
I20260926 06:22:42.155520 17191 ts_manager.cc:194] Registered new tserver with Master: 4b9f9bc12465427b9af617c47dda742e (127.4.202.130:36317)
I20260926 06:22:42.156009 17191 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.130@KRBTEST.COM'} at 127.4.202.130:40241
WARNING: no policy specified for kudu/127.4.202.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
I20260926 06:22:42.181664 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.202.148:36023/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.131:0
--local_ip_for_outbound_sockets=127.4.202.131
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:43269
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:41935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:22:42.301790 17546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:22:42.302023 17546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:22:42.302060 17546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:22:42.303244 17546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:22:42.303318 17546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:22:42.303362 17546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.131
I20260926 06:22:42.304760 17546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:41935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.4.202.148:36023/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:43269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.131
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:22:42.305018 17546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:22:42.305240 17546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:22:42.311348 17554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:42.311352 17552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:42.311364 17551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:22:42.312707 17546 server_base.cc:1034] running on GCE node
I20260926 06:22:42.312947 17546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:22:42.313218 17546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:22:42.314373 17546 hybrid_clock.cc:648] HybridClock initialized: now 1790403762314356 us; error 36 us; skew 500 ppm
Sep 26 06:22:42 dist-test-slave-pd0t krb5kdc[16715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403762, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:22:42.317435 17546 init.cc:377] Logged in from keytab as kudu/127.4.202.131@KRBTEST.COM (short username kudu)
I20260926 06:22:42.317931 17546 webserver.cc:460] Webserver started at http://127.4.202.131:38681/ using document root <none> and password file <none>
I20260926 06:22:42.318213 17546 fs_manager.cc:359] Metadata directory not provided
I20260926 06:22:42.318270 17546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:22:42.318403 17546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 06:22:42 dist-test-slave-pd0t krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403762, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for HTTP/127.4.202.148@KRBTEST.COM
I20260926 06:22:42.335144 17546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-2/data/instance:
uuid: "6213885d7cd04f5ea3c097bc97f2526b"
format_stamp: "Formatted at 2026-09-26 06:22:42 on dist-test-slave-pd0t"
server_key: "008ee6e3417eeff858d681bc4861c893"
server_key_iv: "2330578f49ef22a0724e4eec5553e8ed"
server_key_version: "kuduclusterkey@0"
I20260926 06:22:42.335613 17546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance:
uuid: "6213885d7cd04f5ea3c097bc97f2526b"
format_stamp: "Formatted at 2026-09-26 06:22:42 on dist-test-slave-pd0t"
server_key: "008ee6e3417eeff858d681bc4861c893"
server_key_iv: "2330578f49ef22a0724e4eec5553e8ed"
server_key_version: "kuduclusterkey@0"
I20260926 06:22:42.338838 17546 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260926 06:22:42.349782 17561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:42.349975 17546 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:22:42.350063 17546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-2/wal
uuid: "6213885d7cd04f5ea3c097bc97f2526b"
format_stamp: "Formatted at 2026-09-26 06:22:42 on dist-test-slave-pd0t"
server_key: "008ee6e3417eeff858d681bc4861c893"
server_key_iv: "2330578f49ef22a0724e4eec5553e8ed"
server_key_version: "kuduclusterkey@0"
I20260926 06:22:42.350138 17546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:22:42.373425 17546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:22:42.379652 17546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:22:42.379838 17546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:22:42.380108 17546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:22:42.380483 17546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:22:42.380537 17546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:42.380576 17546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:22:42.380601 17546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:42.387791 17546 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.131:42947
I20260926 06:22:42.388229 17546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
I20260926 06:22:42.389361 17674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.131:42947 every 8 connection(s)
I20260926 06:22:42.391745 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 17546
I20260926 06:22:42.391835 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance
I20260926 06:22:42.392009 4906 external_mini_cluster.cc:1428] Setting key 2aa4ccc96b54c5d272fcab96624be2b9
Sep 26 06:22:42 dist-test-slave-pd0t krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403762, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:22:42.398952 17406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:42.389121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:52399 (local address 127.4.202.190:43269)
0926 06:22:42.389165 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:42.389167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:42.389702 (+ 535us) server_negotiation.cc:407] Connection header received
0926 06:22:42.390233 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:42.390234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:42.390247 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:42.390308 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:42.390931 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:42.391437 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:42.392913 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:42.393113 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:42.393201 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:42.395516 (+ 2315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:42.395518 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:42.395518 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:42.395526 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:42.397331 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:42.397814 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:42.397815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:42.397815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:42.397881 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:42.398189 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:42.398190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:42.398190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:42.398338 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:42.398412 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:42.398761 (+ 349us) server_negotiation.cc:299] Negotiation successful
0926 06:22:42.398845 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 06:22:42.399449 17677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:42.389135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:43269 (local address 127.4.202.131:52399)
0926 06:22:42.389578 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:42.389622 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:42.390129 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:42.390390 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:42.390394 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:42.390619 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:22:42.390827 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:42.390831 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:42.391533 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:42.391534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:42.392799 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:42.392802 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:42.393224 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:42.393226 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:42.393355 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:42.393868 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:22:42.393877 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:22:42.395415 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:22:42.397422 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:42.397428 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:42.397431 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:42.397706 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:42.397967 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:42.397968 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:42.397968 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:42.398105 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:42.398417 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:22:42.398423 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:22:42.398573 (+ 150us) client_negotiation.cc:769] Sending connection context
0926 06:22:42.399075 (+ 502us) client_negotiation.cc:240] Negotiation successful
0926 06:22:42.399230 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":222,"threads_started":1}
I20260926 06:22:42.399864 17675 heartbeater.cc:347] Connected to a master server at 127.4.202.190:43269
I20260926 06:22:42.399955 17675 heartbeater.cc:464] Registering TS with master...
I20260926 06:22:42.400169 17675 heartbeater.cc:511] Master 127.4.202.190:43269 requested a full tablet report, sending...
I20260926 06:22:42.400481 17191 ts_manager.cc:194] Registered new tserver with Master: 6213885d7cd04f5ea3c097bc97f2526b (127.4.202.131:42947)
I20260926 06:22:42.400978 17191 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.131@KRBTEST.COM'} at 127.4.202.131:52399
I20260926 06:22:42.403512 4906 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:22:42.404114 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 17272
I20260926 06:22:42.409914 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 17409
I20260926 06:22:42.413975 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 17546
I20260926 06:22:42.417883 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 17159
I20260926 06:22:42.422657 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:43269
--webserver_interface=127.4.202.190
--webserver_port=37483
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:41935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:43269
--ranger_config_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.202.148:36023/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.202.148:36023/kms with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:22:42.501374 17678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:22:42.501564 17678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:22:42.501585 17678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:22:42.502782 17678 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:22:42.502823 17678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:22:42.502837 17678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:22:42.502851 17678 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:22:42.502867 17678 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:22:42.504390 17678 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:41935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.202.148:36023/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:43269
--ranger_config_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:43269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=37483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:22:42.504612 17678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:22:42.504832 17678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:22:42.509753 17686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:42.509904 17684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:42.509753 17683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:22:42.509966 17678 server_base.cc:1034] running on GCE node
I20260926 06:22:42.510226 17678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:22:42.510452 17678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:22:42.511617 17678 hybrid_clock.cc:648] HybridClock initialized: now 1790403762511616 us; error 44 us; skew 500 ppm
Sep 26 06:22:42 dist-test-slave-pd0t krb5kdc[16715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403762, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:22:42.514062 17678 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:22:42.514496 17678 webserver.cc:460] Webserver started at http://127.4.202.190:37483/ using document root <none> and password file <none>
I20260926 06:22:42.514727 17678 fs_manager.cc:359] Metadata directory not provided
I20260926 06:22:42.514779 17678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 06:22:42 dist-test-slave-pd0t krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403762, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for HTTP/127.4.202.148@KRBTEST.COM
I20260926 06:22:42.565625 17678 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:22:42.566316 17694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:42.566464 17678 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:22:42.566610 17678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "eba7d6e554794125855840f041ebba50"
format_stamp: "Formatted at 2026-09-26 06:22:40 on dist-test-slave-pd0t"
server_key: "45576aa682303c299a9c10a69762b962"
server_key_iv: "65a1eb7eb35c9ae5a30f3d592dc59eac"
server_key_version: "kuduclusterkey@0"
I20260926 06:22:42.566957 17678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:22:42.584070 17678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:22:42.586091 17678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:22:42.586248 17678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:22:42.590313 17678 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:43269
I20260926 06:22:42.590358 17746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:43269 every 8 connection(s)
I20260926 06:22:42.590713 17678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:22:42.591828 17747 sys_catalog.cc:263] Verifying existing consensus state
I20260926 06:22:42.592516 17747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50: Bootstrap starting.
I20260926 06:22:42.597666 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 17678
I20260926 06:22:42.598002 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.202.148:36023/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.129:39293
--local_ip_for_outbound_sockets=127.4.202.129
--tserver_master_addrs=127.4.202.190:43269
--webserver_port=38653
--webserver_interface=127.4.202.129
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:41935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.202.148:36023/kms with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:22:42.600839 17747 log.cc:824] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50: Log is configured to *not* fsync() on all Append() calls
I20260926 06:22:42.602463 17747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 06:22:42.602763 17747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50: Bootstrap complete.
I20260926 06:22:42.605005 17747 raft_consensus.cc:348] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba7d6e554794125855840f041ebba50" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43269 } }
I20260926 06:22:42.605219 17747 raft_consensus.cc:729] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eba7d6e554794125855840f041ebba50, State: Initialized, Role: FOLLOWER
I20260926 06:22:42.605376 17747 consensus_queue.cc:260] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba7d6e554794125855840f041ebba50" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43269 } }
I20260926 06:22:42.605443 17747 raft_consensus.cc:388] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:22:42.605517 17747 raft_consensus.cc:482] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:22:42.605580 17747 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [term 1 FOLLOWER]: Advancing to term 2
I20260926 06:22:42.606560 17747 raft_consensus.cc:504] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba7d6e554794125855840f041ebba50" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43269 } }
I20260926 06:22:42.606698 17747 leader_election.cc:302] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eba7d6e554794125855840f041ebba50; no voters:
I20260926 06:22:42.606899 17747 leader_election.cc:288] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 06:22:42.606930 17751 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 06:22:42.607033 17751 raft_consensus.cc:686] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [term 2 LEADER]: Becoming Leader. State: Replica: eba7d6e554794125855840f041ebba50, State: Running, Role: LEADER
I20260926 06:22:42.607121 17751 consensus_queue.cc:237] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba7d6e554794125855840f041ebba50" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43269 } }
I20260926 06:22:42.607151 17747 sys_catalog.cc:564] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:22:42.607565 17753 sys_catalog.cc:455] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eba7d6e554794125855840f041ebba50. Latest consensus state: current_term: 2 leader_uuid: "eba7d6e554794125855840f041ebba50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba7d6e554794125855840f041ebba50" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43269 } } }
I20260926 06:22:42.607733 17753 sys_catalog.cc:458] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [sys.catalog]: This master's current role is: LEADER
I20260926 06:22:42.608117 17752 sys_catalog.cc:455] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "eba7d6e554794125855840f041ebba50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba7d6e554794125855840f041ebba50" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43269 } } }
I20260926 06:22:42.608178 17759 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:22:42.608186 17752 sys_catalog.cc:458] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50 [sys.catalog]: This master's current role is: LEADER
I20260926 06:22:42.608697 17759 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:22:42.608832 17759 catalog_manager.cc:1259] Loaded cluster ID: 57e2cac7562f451bbb427d1367a66580
I20260926 06:22:42.608868 17759 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:22:42.609086 17747 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 06:22:42.609963 17759 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:22:42.610118 17759 catalog_manager.cc:5879] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50: Loaded TSK: 0
I20260926 06:22:42.610327 17759 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 06:22:42.704411 17749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:22:42.704605 17749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:22:42.704640 17749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:22:42.705715 17749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:22:42.705776 17749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:22:42.705818 17749 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.129
I20260926 06:22:42.708297 17749 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:41935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.202.148:36023/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.129:39293
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.202.129
--webserver_port=38653
--tserver_master_addrs=127.4.202.190:43269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17749
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.129
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:22:42.708624 17749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:22:42.708901 17749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:22:42.715104 17787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:42.715147 17785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:42.715144 17784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:22:42.715968 17749 server_base.cc:1034] running on GCE node
I20260926 06:22:42.716195 17749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:22:42.716467 17749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:22:42.717631 17749 hybrid_clock.cc:648] HybridClock initialized: now 1790403762717610 us; error 33 us; skew 500 ppm
Sep 26 06:22:42 dist-test-slave-pd0t krb5kdc[16715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403762, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:22:42.720620 17749 init.cc:377] Logged in from keytab as kudu/127.4.202.129@KRBTEST.COM (short username kudu)
I20260926 06:22:42.721113 17749 webserver.cc:460] Webserver started at http://127.4.202.129:38653/ using document root <none> and password file <none>
I20260926 06:22:42.721379 17749 fs_manager.cc:359] Metadata directory not provided
I20260926 06:22:42.721472 17749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 06:22:42 dist-test-slave-pd0t krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403762, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for HTTP/127.4.202.148@KRBTEST.COM
I20260926 06:22:42.747008 17749 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.000s
I20260926 06:22:42.749922 17795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:42.750128 17749 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 06:22:42.750207 17749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-0/wal
uuid: "03a2050bb55341429a92ec2c53883730"
format_stamp: "Formatted at 2026-09-26 06:22:41 on dist-test-slave-pd0t"
server_key: "1b391592c078118fa82924f34794556c"
server_key_iv: "7ac6ce078c0cc8fb60636b632dc0c7e6"
server_key_version: "kuduclusterkey@0"
I20260926 06:22:42.750746 17749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:22:42.765161 17749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:22:42.767369 17749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:22:42.767525 17749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:22:42.767778 17749 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:22:42.768132 17749 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:22:42.768187 17749 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:42.768239 17749 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:22:42.768265 17749 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:42.775635 17749 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.129:39293
I20260926 06:22:42.775722 17908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.129:39293 every 8 connection(s)
I20260926 06:22:42.776072 17749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
Sep 26 06:22:42 dist-test-slave-pd0t krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403762, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:22:42.785254 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 17749
I20260926 06:22:42.785543 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.202.148:36023/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.130:36317
--local_ip_for_outbound_sockets=127.4.202.130
--tserver_master_addrs=127.4.202.190:43269
--webserver_port=33277
--webserver_interface=127.4.202.130
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:41935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.202.148:36023/kms with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:22:42.786485 17912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:42.776873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:40547 (local address 127.4.202.190:43269)
0926 06:22:42.777617 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:42.777624 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:42.777649 (+ 25us) server_negotiation.cc:407] Connection header received
0926 06:22:42.777767 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:42.777770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:42.777809 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:42.777940 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:42.778476 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:42.779395 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:42.780393 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:42.780872 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:42.781007 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:42.782979 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:42.782983 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:42.782984 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:42.782995 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:42.784635 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:42.785078 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:42.785080 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:42.785081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:42.785179 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:42.785504 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:42.785505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:42.785505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:42.785795 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:42.785912 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:42.786188 (+ 276us) server_negotiation.cc:299] Negotiation successful
0926 06:22:42.786287 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":697,"thread_start_us":80,"threads_started":1}
I20260926 06:22:42.786499 17911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:42.776787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:43269 (local address 127.4.202.129:40547)
0926 06:22:42.776960 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:42.777000 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:42.777461 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:42.777979 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:42.777983 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:42.778162 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:22:42.778351 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:42.778356 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:42.779495 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:42.779496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:42.780277 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:42.780280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:42.780994 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:42.780995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:42.781114 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:42.781610 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:22:42.781617 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:22:42.782872 (+ 1255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:22:42.784734 (+ 1862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:42.784740 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:42.784742 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:42.784980 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:42.785251 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:42.785251 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:42.785252 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:42.785425 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:42.785897 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:22:42.785902 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:22:42.786030 (+ 128us) client_negotiation.cc:769] Sending connection context
0926 06:22:42.786138 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 06:22:42.786287 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":107,"threads_started":1}
I20260926 06:22:42.787796 17909 heartbeater.cc:347] Connected to a master server at 127.4.202.190:43269
I20260926 06:22:42.787895 17909 heartbeater.cc:464] Registering TS with master...
I20260926 06:22:42.788080 17909 heartbeater.cc:511] Master 127.4.202.190:43269 requested a full tablet report, sending...
W20260926 06:22:42.789001 17909 heartbeater.cc:644] Failed to heartbeat to 127.4.202.190:43269 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 06:22:42.789088 17909 heartbeater.cc:464] Registering TS with master...
I20260926 06:22:42.789150 17909 heartbeater.cc:511] Master 127.4.202.190:43269 requested a full tablet report, sending...
I20260926 06:22:42.789494 17909 heartbeater.cc:464] Registering TS with master...
I20260926 06:22:42.789568 17909 heartbeater.cc:511] Master 127.4.202.190:43269 requested a full tablet report, sending...
W20260926 06:22:42.789845 17909 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 06:22:42.884876 17913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:22:42.885123 17913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:22:42.885156 17913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:22:42.886792 17913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:22:42.886878 17913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:22:42.886916 17913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.130
I20260926 06:22:42.888985 17913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:41935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.202.148:36023/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.130:36317
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.202.130
--webserver_port=33277
--tserver_master_addrs=127.4.202.190:43269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.130
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:22:42.889215 17913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:22:42.889446 17913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:22:42.897742 17918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:42.897742 17919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:42.898200 17921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:22:42.898659 17913 server_base.cc:1034] running on GCE node
I20260926 06:22:42.898921 17913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:22:42.899173 17913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:22:42.900331 17913 hybrid_clock.cc:648] HybridClock initialized: now 1790403762900309 us; error 36 us; skew 500 ppm
Sep 26 06:22:42 dist-test-slave-pd0t krb5kdc[16715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403762, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:22:42.902915 17913 init.cc:377] Logged in from keytab as kudu/127.4.202.130@KRBTEST.COM (short username kudu)
I20260926 06:22:42.903412 17913 webserver.cc:460] Webserver started at http://127.4.202.130:33277/ using document root <none> and password file <none>
I20260926 06:22:42.903658 17913 fs_manager.cc:359] Metadata directory not provided
I20260926 06:22:42.903733 17913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 06:22:42 dist-test-slave-pd0t krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403762, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for HTTP/127.4.202.148@KRBTEST.COM
I20260926 06:22:42.931033 17913 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.000s sys 0.001s
I20260926 06:22:42.934482 17929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:42.934672 17913 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 06:22:42.934751 17913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-1/wal
uuid: "4b9f9bc12465427b9af617c47dda742e"
format_stamp: "Formatted at 2026-09-26 06:22:42 on dist-test-slave-pd0t"
server_key: "62d56f1a70376d5e56bdd7089b9f5114"
server_key_iv: "1ab27a5ad22a10c7435213e81041e5c6"
server_key_version: "kuduclusterkey@0"
I20260926 06:22:42.935238 17913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:22:42.962191 17913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:22:42.964560 17913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:22:42.964696 17913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:22:42.964921 17913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:22:42.965262 17913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:22:42.965301 17913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:42.965328 17913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:22:42.965344 17913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:42.971396 17913 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.130:36317
I20260926 06:22:42.971788 17913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
I20260926 06:22:42.975769 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 17913
I20260926 06:22:42.976043 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.202.148:36023/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.131:42947
--local_ip_for_outbound_sockets=127.4.202.131
--tserver_master_addrs=127.4.202.190:43269
--webserver_port=38681
--webserver_interface=127.4.202.131
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:41935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.202.148:36023/kms with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:22:42.982986 18042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.130:36317 every 8 connection(s)
Sep 26 06:22:42 dist-test-slave-pd0t krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403762, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:22:42.998762 17912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:42.983600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:41431 (local address 127.4.202.190:43269)
0926 06:22:42.983643 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:42.983645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:42.984298 (+ 653us) server_negotiation.cc:407] Connection header received
0926 06:22:42.985014 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:42.985016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:42.985032 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:42.985069 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:42.985911 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:42.986740 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:42.988202 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:42.988508 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:42.988557 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:42.995391 (+ 6834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:42.995392 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:42.995393 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:42.995401 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:42.997109 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:42.997633 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:42.997635 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:42.997635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:42.997700 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:42.998014 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:42.998014 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:42.998015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:42.998140 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:42.998206 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:42.998565 (+ 359us) server_negotiation.cc:299] Negotiation successful
0926 06:22:42.998664 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 06:22:42.999260 18046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:42.983237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:43269 (local address 127.4.202.130:41431)
0926 06:22:42.984145 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:42.984191 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:42.984881 (+ 690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:42.985261 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:42.985266 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:42.985510 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:22:42.985792 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:42.985800 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:42.986851 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:42.986852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:42.988099 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:42.988102 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:42.988684 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:42.988685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:42.988827 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:42.991287 (+ 2460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:22:42.991298 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:22:42.995270 (+ 3972us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:22:42.997240 (+ 1970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:42.997246 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:42.997248 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:42.997546 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:42.997804 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:42.997805 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:42.997806 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:42.997939 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:42.998218 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:22:42.998225 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:22:42.998402 (+ 177us) client_negotiation.cc:769] Sending connection context
0926 06:22:42.998529 (+ 127us) client_negotiation.cc:240] Negotiation successful
0926 06:22:42.998738 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":862,"thread_start_us":107,"threads_started":1}
I20260926 06:22:42.999754 18043 heartbeater.cc:347] Connected to a master server at 127.4.202.190:43269
I20260926 06:22:42.999861 18043 heartbeater.cc:464] Registering TS with master...
I20260926 06:22:43.000041 18043 heartbeater.cc:511] Master 127.4.202.190:43269 requested a full tablet report, sending...
W20260926 06:22:43.000701 18043 heartbeater.cc:644] Failed to heartbeat to 127.4.202.190:43269 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 06:22:43.000768 18043 heartbeater.cc:464] Registering TS with master...
I20260926 06:22:43.000836 18043 heartbeater.cc:511] Master 127.4.202.190:43269 requested a full tablet report, sending...
I20260926 06:22:43.003130 18043 heartbeater.cc:464] Registering TS with master...
I20260926 06:22:43.003208 18043 heartbeater.cc:511] Master 127.4.202.190:43269 requested a full tablet report, sending...
W20260926 06:22:43.003540 18043 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 06:22:43.077868 18045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:22:43.078079 18045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:22:43.078117 18045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:22:43.079375 18045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:22:43.079424 18045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:22:43.079463 18045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.131
I20260926 06:22:43.081624 18045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:41935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.202.148:36023/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.131:42947
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.202.131
--webserver_port=38681
--tserver_master_addrs=127.4.202.190:43269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.131
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:22:43.081861 18045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:22:43.082122 18045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:22:43.087528 18054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:22:43.087653 18045 server_base.cc:1034] running on GCE node
W20260926 06:22:43.087816 18051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:43.087960 18052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:22:43.088243 18045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:22:43.088523 18045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:22:43.089681 18045 hybrid_clock.cc:648] HybridClock initialized: now 1790403763089670 us; error 26 us; skew 500 ppm
Sep 26 06:22:43 dist-test-slave-pd0t krb5kdc[16715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403763, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:22:43.092221 18045 init.cc:377] Logged in from keytab as kudu/127.4.202.131@KRBTEST.COM (short username kudu)
I20260926 06:22:43.092757 18045 webserver.cc:460] Webserver started at http://127.4.202.131:38681/ using document root <none> and password file <none>
I20260926 06:22:43.093009 18045 fs_manager.cc:359] Metadata directory not provided
I20260926 06:22:43.093087 18045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 06:22:43 dist-test-slave-pd0t krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403763, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for HTTP/127.4.202.148@KRBTEST.COM
I20260926 06:22:43.111657 18045 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:22:43.112634 18063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:43.112783 18045 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:22:43.112866 18045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-2/wal
uuid: "6213885d7cd04f5ea3c097bc97f2526b"
format_stamp: "Formatted at 2026-09-26 06:22:42 on dist-test-slave-pd0t"
server_key: "008ee6e3417eeff858d681bc4861c893"
server_key_iv: "2330578f49ef22a0724e4eec5553e8ed"
server_key_version: "kuduclusterkey@0"
I20260926 06:22:43.113317 18045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:22:43.130142 18045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:22:43.132390 18045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:22:43.132527 18045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:22:43.132707 18045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:22:43.133023 18045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:22:43.133075 18045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:43.133102 18045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:22:43.133119 18045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:43.139849 18045 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.131:42947
I20260926 06:22:43.139914 18176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.131:42947 every 8 connection(s)
I20260926 06:22:43.140208 18045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
I20260926 06:22:43.143123 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 18045
Sep 26 06:22:43 dist-test-slave-pd0t krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403763, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:22:43.149799 17912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:43.140806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:33673 (local address 127.4.202.190:43269)
0926 06:22:43.140857 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:43.140859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:43.141120 (+ 261us) server_negotiation.cc:407] Connection header received
0926 06:22:43.141646 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:43.141647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:43.141660 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:43.141713 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:43.142223 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:43.142787 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:43.144193 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:43.144559 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:43.144700 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:43.146879 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:43.146881 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:43.146881 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:43.146889 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:43.148347 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:43.148737 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:43.148739 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:43.148739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:43.148795 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:43.149055 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:43.149056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:43.149057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:43.149178 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:43.149252 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:43.149600 (+ 348us) server_negotiation.cc:299] Negotiation successful
0926 06:22:43.149695 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 06:22:43.149906 18179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:43.140817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:43269 (local address 127.4.202.131:33673)
0926 06:22:43.141003 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:43.141042 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:43.141534 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:43.141783 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:43.141786 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:43.141943 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:22:43.142111 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:43.142115 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:43.142880 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:43.142881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:43.144085 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:43.144088 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:43.144784 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:43.144785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:43.144925 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:43.145449 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:22:43.145459 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:22:43.146784 (+ 1325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:22:43.148429 (+ 1645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:43.148435 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:43.148437 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:43.148662 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:43.148881 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:43.148881 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:43.148882 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:43.148989 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:43.149257 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:22:43.149263 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:22:43.149444 (+ 181us) client_negotiation.cc:769] Sending connection context
0926 06:22:43.149558 (+ 114us) client_negotiation.cc:240] Negotiation successful
0926 06:22:43.149696 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":103,"threads_started":1}
I20260926 06:22:43.150705 18177 heartbeater.cc:347] Connected to a master server at 127.4.202.190:43269
I20260926 06:22:43.151036 18177 heartbeater.cc:464] Registering TS with master...
I20260926 06:22:43.151234 18177 heartbeater.cc:511] Master 127.4.202.190:43269 requested a full tablet report, sending...
W20260926 06:22:43.152017 18177 heartbeater.cc:644] Failed to heartbeat to 127.4.202.190:43269 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 06:22:43.152092 18177 heartbeater.cc:464] Registering TS with master...
I20260926 06:22:43.152163 18177 heartbeater.cc:511] Master 127.4.202.190:43269 requested a full tablet report, sending...
I20260926 06:22:43.152563 18177 heartbeater.cc:464] Registering TS with master...
I20260926 06:22:43.152638 18177 heartbeater.cc:511] Master 127.4.202.190:43269 requested a full tablet report, sending...
W20260926 06:22:43.152926 18177 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 06:22:43.153404 18180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:43.143580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50954 (local address 127.4.202.190:43269)
0926 06:22:43.144239 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:43.144240 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:43.144549 (+ 309us) server_negotiation.cc:407] Connection header received
0926 06:22:43.144577 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:43.144578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:43.144591 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:43.144663 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:43.145468 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:43.146237 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:43.148523 (+ 2286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:43.148838 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:43.148903 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:43.150537 (+ 1634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:43.150539 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:43.150539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:43.150546 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:43.152125 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:43.152562 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:43.152563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:43.152564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:43.152624 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:43.152892 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:43.152893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:43.152893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:43.153016 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:43.153173 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:43.153211 (+ 38us) server_negotiation.cc:299] Negotiation successful
0926 06:22:43.153288 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":68,"threads_started":1}
Sep 26 06:22:43 dist-test-slave-pd0t krb5kdc[16715](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790403763, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:22:43.790033 17909 heartbeater.cc:464] Registering TS with master...
I20260926 06:22:43.790191 17909 heartbeater.cc:511] Master 127.4.202.190:43269 requested a full tablet report, sending...
I20260926 06:22:43.790616 17711 ts_manager.cc:194] Registered new tserver with Master: 03a2050bb55341429a92ec2c53883730 (127.4.202.129:39293)
I20260926 06:22:43.791618 17711 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.129@KRBTEST.COM'} at 127.4.202.129:40547
I20260926 06:22:44.003702 18043 heartbeater.cc:464] Registering TS with master...
I20260926 06:22:44.003845 18043 heartbeater.cc:511] Master 127.4.202.190:43269 requested a full tablet report, sending...
I20260926 06:22:44.004230 17711 ts_manager.cc:194] Registered new tserver with Master: 4b9f9bc12465427b9af617c47dda742e (127.4.202.130:36317)
I20260926 06:22:44.004773 17711 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.130@KRBTEST.COM'} at 127.4.202.130:41431
Sep 26 06:22:44 dist-test-slave-pd0t krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403763, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for HTTP/127.4.202.148@KRBTEST.COM
I20260926 06:22:44.153090 18177 heartbeater.cc:464] Registering TS with master...
I20260926 06:22:44.153231 18177 heartbeater.cc:511] Master 127.4.202.190:43269 requested a full tablet report, sending...
I20260926 06:22:44.153537 17711 ts_manager.cc:194] Registered new tserver with Master: 6213885d7cd04f5ea3c097bc97f2526b (127.4.202.131:42947)
I20260926 06:22:44.154074 17711 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.131@KRBTEST.COM'} at 127.4.202.131:33673
I20260926 06:22:44.160933 4906 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:22:44.170403 18199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:44.162086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50962 (local address 127.4.202.190:43269)
0926 06:22:44.162209 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:44.162212 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:44.162321 (+ 109us) server_negotiation.cc:407] Connection header received
0926 06:22:44.162422 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:44.162423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:44.162435 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:44.162507 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:44.163202 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:44.163999 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:44.165046 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:44.165374 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:44.165439 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:44.166977 (+ 1538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:44.166979 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:44.166979 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:44.166987 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:44.169020 (+ 2033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:44.169446 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:44.169447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:44.169447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:44.169513 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:44.169750 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:44.169751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:44.169751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:44.169877 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:44.169982 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:44.170132 (+ 150us) server_negotiation.cc:299] Negotiation successful
0926 06:22:44.170238 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":67,"threads_started":1}
I20260926 06:22:44.172662 17711 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50962:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:22:44.174830 17711 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:22:44.182861 18206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:44.177953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:36317 (local address 127.0.0.1:53776)
0926 06:22:44.178465 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:44.178494 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:44.178559 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:44.179045 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:44.179048 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:44.179050 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:22:44.179104 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:44.179105 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:44.180328 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:44.180329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:44.181552 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:44.181555 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:44.182537 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:44.182538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:44.182639 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:44.182654 (+ 15us) client_negotiation.cc:769] Sending connection context
0926 06:22:44.182724 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 06:22:44.182770 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":69,"threads_started":1}
I20260926 06:22:44.183240 18207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:44.178052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53776 (local address 127.4.202.130:36317)
0926 06:22:44.178412 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:44.178417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:44.178627 (+ 210us) server_negotiation.cc:407] Connection header received
0926 06:22:44.178862 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:44.178865 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:44.178884 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:44.178956 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:22:44.179171 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:44.180231 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:44.181645 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:44.182379 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:44.182452 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:44.182502 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:44.183118 (+ 616us) server_negotiation.cc:299] Negotiation successful
0926 06:22:44.183163 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":49,"threads_started":1}
I20260926 06:22:44.184018 17965 tablet_service.cc:1467] Processing CreateTablet for tablet 89b535d0d71d44208207d19532d4c232 (DEFAULT_TABLE table=test-table [id=38897f9ea6ad4e78b00b97300ca51f05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:22:44.184320 17965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89b535d0d71d44208207d19532d4c232. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:22:44.186501 18208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:44.178093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:39293 (local address 127.0.0.1:50020)
0926 06:22:44.178700 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:44.178709 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:44.178749 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:44.182993 (+ 4244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:44.182994 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:44.182996 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:22:44.183030 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:44.183031 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:44.184211 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:44.184211 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:44.185299 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:44.185301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:44.186272 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:44.186273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:44.186330 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:44.186334 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:22:44.186394 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 06:22:44.186430 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":56,"threads_started":1}
I20260926 06:22:44.186972 18209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:44.178241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50020 (local address 127.4.202.129:39293)
0926 06:22:44.182777 (+ 4536us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:44.182781 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:44.182791 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:22:44.182823 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:44.182825 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:44.182840 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:44.182909 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:22:44.183098 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:44.184117 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:44.185441 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:44.186175 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:44.186249 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:44.186299 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:44.186770 (+ 471us) server_negotiation.cc:299] Negotiation successful
0926 06:22:44.186822 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4494,"thread_start_us":64,"threads_started":1}
I20260926 06:22:44.187830 17843 tablet_service.cc:1467] Processing CreateTablet for tablet 89b535d0d71d44208207d19532d4c232 (DEFAULT_TABLE table=test-table [id=38897f9ea6ad4e78b00b97300ca51f05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:22:44.188143 17843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89b535d0d71d44208207d19532d4c232. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:22:44.190735 18210 tablet_bootstrap.cc:492] T 89b535d0d71d44208207d19532d4c232 P 4b9f9bc12465427b9af617c47dda742e: Bootstrap starting.
I20260926 06:22:44.191397 18210 tablet_bootstrap.cc:654] T 89b535d0d71d44208207d19532d4c232 P 4b9f9bc12465427b9af617c47dda742e: Neither blocks nor log segments found. Creating new log.
I20260926 06:22:44.191743 18210 log.cc:824] T 89b535d0d71d44208207d19532d4c232 P 4b9f9bc12465427b9af617c47dda742e: Log is configured to *not* fsync() on all Append() calls
I20260926 06:22:44.192301 18211 tablet_bootstrap.cc:492] T 89b535d0d71d44208207d19532d4c232 P 03a2050bb55341429a92ec2c53883730: Bootstrap starting.
I20260926 06:22:44.192947 18211 tablet_bootstrap.cc:654] T 89b535d0d71d44208207d19532d4c232 P 03a2050bb55341429a92ec2c53883730: Neither blocks nor log segments found. Creating new log.
I20260926 06:22:44.193302 18211 log.cc:824] T 89b535d0d71d44208207d19532d4c232 P 03a2050bb55341429a92ec2c53883730: Log is configured to *not* fsync() on all Append() calls
I20260926 06:22:44.193996 18211 tablet_bootstrap.cc:492] T 89b535d0d71d44208207d19532d4c232 P 03a2050bb55341429a92ec2c53883730: No bootstrap required, opened a new log
I20260926 06:22:44.194051 18211 ts_tablet_manager.cc:1397] T 89b535d0d71d44208207d19532d4c232 P 03a2050bb55341429a92ec2c53883730: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:22:44.194543 18204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:44.177703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:42947 (local address 127.0.0.1:54436)
0926 06:22:44.178457 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:44.178494 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:44.178545 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:44.186530 (+ 7985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:44.186531 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:44.186533 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:22:44.186572 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:44.186573 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:44.192200 (+ 5627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:44.192201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:44.193331 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:44.193334 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:44.194318 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:44.194319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:44.194377 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:44.194380 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:22:44.194431 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 06:22:44.194468 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":85,"threads_started":1}
I20260926 06:22:44.194762 18205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:44.177817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54436 (local address 127.4.202.131:42947)
0926 06:22:44.186372 (+ 8555us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:44.186376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:44.186385 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:22:44.186416 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:44.186417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:44.186432 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:44.186669 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:22:44.186756 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:44.192109 (+ 5353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:44.193435 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:44.194157 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:44.194230 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:44.194285 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:44.194662 (+ 377us) server_negotiation.cc:299] Negotiation successful
0926 06:22:44.194698 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8523,"thread_start_us":72,"threads_started":1}
I20260926 06:22:44.196237 18211 raft_consensus.cc:348] T 89b535d0d71d44208207d19532d4c232 P 03a2050bb55341429a92ec2c53883730 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6213885d7cd04f5ea3c097bc97f2526b" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 42947 } } peers { permanent_uuid: "4b9f9bc12465427b9af617c47dda742e" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36317 } } peers { permanent_uuid: "03a2050bb55341429a92ec2c53883730" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 39293 } }
I20260926 06:22:44.196379 18211 raft_consensus.cc:374] T 89b535d0d71d44208207d19532d4c232 P 03a2050bb55341429a92ec2c53883730 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:22:44.196406 18211 raft_consensus.cc:729] T 89b535d0d71d44208207d19532d4c232 P 03a2050bb55341429a92ec2c53883730 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03a2050bb55341429a92ec2c53883730, State: Initialized, Role: FOLLOWER
I20260926 06:22:44.196511 18211 consensus_queue.cc:260] T 89b535d0d71d44208207d19532d4c232 P 03a2050bb55341429a92ec2c53883730 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6213885d7cd04f5ea3c097bc97f2526b" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 42947 } } peers { permanent_uuid: "4b9f9bc12465427b9af617c47dda742e" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36317 } } peers { permanent_uuid: "03a2050bb55341429a92ec2c53883730" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 39293 } }
I20260926 06:22:44.196715 18211 ts_tablet_manager.cc:1428] T 89b535d0d71d44208207d19532d4c232 P 03a2050bb55341429a92ec2c53883730: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 06:22:44.197096 18210 tablet_bootstrap.cc:492] T 89b535d0d71d44208207d19532d4c232 P 4b9f9bc12465427b9af617c47dda742e: No bootstrap required, opened a new log
I20260926 06:22:44.197161 18210 ts_tablet_manager.cc:1397] T 89b535d0d71d44208207d19532d4c232 P 4b9f9bc12465427b9af617c47dda742e: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260926 06:22:44.197825 18111 tablet_service.cc:1467] Processing CreateTablet for tablet 89b535d0d71d44208207d19532d4c232 (DEFAULT_TABLE table=test-table [id=38897f9ea6ad4e78b00b97300ca51f05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:22:44.198101 18111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89b535d0d71d44208207d19532d4c232. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:22:44.199323 18210 raft_consensus.cc:348] T 89b535d0d71d44208207d19532d4c232 P 4b9f9bc12465427b9af617c47dda742e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6213885d7cd04f5ea3c097bc97f2526b" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 42947 } } peers { permanent_uuid: "4b9f9bc12465427b9af617c47dda742e" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36317 } } peers { permanent_uuid: "03a2050bb55341429a92ec2c53883730" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 39293 } }
I20260926 06:22:44.199469 18210 raft_consensus.cc:374] T 89b535d0d71d44208207d19532d4c232 P 4b9f9bc12465427b9af617c47dda742e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:22:44.199502 18210 raft_consensus.cc:729] T 89b535d0d71d44208207d19532d4c232 P 4b9f9bc12465427b9af617c47dda742e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b9f9bc12465427b9af617c47dda742e, State: Initialized, Role: FOLLOWER
I20260926 06:22:44.199608 18210 consensus_queue.cc:260] T 89b535d0d71d44208207d19532d4c232 P 4b9f9bc12465427b9af617c47dda742e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6213885d7cd04f5ea3c097bc97f2526b" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 42947 } } peers { permanent_uuid: "4b9f9bc12465427b9af617c47dda742e" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36317 } } peers { permanent_uuid: "03a2050bb55341429a92ec2c53883730" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 39293 } }
I20260926 06:22:44.199827 18210 ts_tablet_manager.cc:1428] T 89b535d0d71d44208207d19532d4c232 P 4b9f9bc12465427b9af617c47dda742e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 06:22:44.199960 18043 heartbeater.cc:503] Master 127.4.202.190:43269 was elected leader, sending a full tablet report...
I20260926 06:22:44.201573 17909 heartbeater.cc:503] Master 127.4.202.190:43269 was elected leader, sending a full tablet report...
I20260926 06:22:44.202075 18216 tablet_bootstrap.cc:492] T 89b535d0d71d44208207d19532d4c232 P 6213885d7cd04f5ea3c097bc97f2526b: Bootstrap starting.
I20260926 06:22:44.202682 18216 tablet_bootstrap.cc:654] T 89b535d0d71d44208207d19532d4c232 P 6213885d7cd04f5ea3c097bc97f2526b: Neither blocks nor log segments found. Creating new log.
I20260926 06:22:44.203008 18216 log.cc:824] T 89b535d0d71d44208207d19532d4c232 P 6213885d7cd04f5ea3c097bc97f2526b: Log is configured to *not* fsync() on all Append() calls
I20260926 06:22:44.203708 18216 tablet_bootstrap.cc:492] T 89b535d0d71d44208207d19532d4c232 P 6213885d7cd04f5ea3c097bc97f2526b: No bootstrap required, opened a new log
I20260926 06:22:44.203821 18216 ts_tablet_manager.cc:1397] T 89b535d0d71d44208207d19532d4c232 P 6213885d7cd04f5ea3c097bc97f2526b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 06:22:44.206313 18216 raft_consensus.cc:348] T 89b535d0d71d44208207d19532d4c232 P 6213885d7cd04f5ea3c097bc97f2526b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6213885d7cd04f5ea3c097bc97f2526b" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 42947 } } peers { permanent_uuid: "4b9f9bc12465427b9af617c47dda742e" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36317 } } peers { permanent_uuid: "03a2050bb55341429a92ec2c53883730" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 39293 } }
I20260926 06:22:44.206516 18216 raft_consensus.cc:374] T 89b535d0d71d44208207d19532d4c232 P 6213885d7cd04f5ea3c097bc97f2526b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:22:44.206609 18216 raft_consensus.cc:729] T 89b535d0d71d44208207d19532d4c232 P 6213885d7cd04f5ea3c097bc97f2526b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6213885d7cd04f5ea3c097bc97f2526b, State: Initialized, Role: FOLLOWER
I20260926 06:22:44.206947 18216 consensus_queue.cc:260] T 89b535d0d71d44208207d19532d4c232 P 6213885d7cd04f5ea3c097bc97f2526b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6213885d7cd04f5ea3c097bc97f2526b" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 42947 } } peers { permanent_uuid: "4b9f9bc12465427b9af617c47dda742e" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36317 } } peers { permanent_uuid: "03a2050bb55341429a92ec2c53883730" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 39293 } }
I20260926 06:22:44.207263 18216 ts_tablet_manager.cc:1428] T 89b535d0d71d44208207d19532d4c232 P 6213885d7cd04f5ea3c097bc97f2526b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 06:22:44.207376 18177 heartbeater.cc:503] Master 127.4.202.190:43269 was elected leader, sending a full tablet report...
W20260926 06:22:44.233598 18044 tablet.cc:2369] T 89b535d0d71d44208207d19532d4c232 P 4b9f9bc12465427b9af617c47dda742e: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 26 06:22:44 dist-test-slave-pd0t krb5kdc[16715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403763, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for HTTP/127.4.202.148@KRBTEST.COM
W20260926 06:22:44.277403 17910 tablet.cc:2369] T 89b535d0d71d44208207d19532d4c232 P 03a2050bb55341429a92ec2c53883730: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:22:44.388230 18214 raft_consensus.cc:482] T 89b535d0d71d44208207d19532d4c232 P 03a2050bb55341429a92ec2c53883730 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:22:44.388388 18214 raft_consensus.cc:504] T 89b535d0d71d44208207d19532d4c232 P 03a2050bb55341429a92ec2c53883730 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6213885d7cd04f5ea3c097bc97f2526b" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 42947 } } peers { permanent_uuid: "4b9f9bc12465427b9af617c47dda742e" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36317 } } peers { permanent_uuid: "03a2050bb55341429a92ec2c53883730" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 39293 } }
I20260926 06:22:44.388792 18214 leader_election.cc:288] T 89b535d0d71d44208207d19532d4c232 P 03a2050bb55341429a92ec2c53883730 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6213885d7cd04f5ea3c097bc97f2526b (127.4.202.131:42947), 4b9f9bc12465427b9af617c47dda742e (127.4.202.130:36317)
I20260926 06:22:44.393049 18205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:44.389221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:60241 (local address 127.4.202.131:42947)
0926 06:22:44.389268 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:44.389270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:44.389414 (+ 144us) server_negotiation.cc:407] Connection header received
0926 06:22:44.389518 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:44.389519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:44.389529 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:44.389556 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:22:44.389777 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:44.390507 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:44.392061 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:44.392645 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:44.392694 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:44.392703 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:44.392938 (+ 235us) server_negotiation.cc:299] Negotiation successful
0926 06:22:44.392984 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 06:22:44.393339 18221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:44.388982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:42947 (local address 127.4.202.129:60241)
0926 06:22:44.389344 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:44.389356 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:44.389444 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:44.389624 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:44.389624 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:44.389626 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:22:44.389699 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:44.389701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:44.390645 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:44.390646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:44.391951 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:44.391954 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:44.392790 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:44.392791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:44.392859 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:44.392862 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:22:44.393205 (+ 343us) client_negotiation.cc:240] Negotiation successful
0926 06:22:44.393247 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":107,"threads_started":1}
I20260926 06:22:44.393956 18207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:44.389200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:60697 (local address 127.4.202.130:36317)
0926 06:22:44.389244 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:44.389246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:44.389438 (+ 192us) server_negotiation.cc:407] Connection header received
0926 06:22:44.389540 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:44.389541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:44.389552 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:44.389582 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:22:44.389802 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:44.390498 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:44.393013 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:44.393548 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:44.393602 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:44.393610 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:44.393835 (+ 225us) server_negotiation.cc:299] Negotiation successful
0926 06:22:44.393891 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 06:22:44.393981 18131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89b535d0d71d44208207d19532d4c232" candidate_uuid: "03a2050bb55341429a92ec2c53883730" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6213885d7cd04f5ea3c097bc97f2526b" is_pre_election: true
I20260926 06:22:44.394134 18131 raft_consensus.cc:2454] T 89b535d0d71d44208207d19532d4c232 P 6213885d7cd04f5ea3c097bc97f2526b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03a2050bb55341429a92ec2c53883730 in term 0.
W20260926 06:22:44.394260 18178 tablet.cc:2369] T 89b535d0d71d44208207d19532d4c232 P 6213885d7cd04f5ea3c097bc97f2526b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:22:44.394404 17796 leader_election.cc:302] T 89b535d0d71d44208207d19532d4c232 P 03a2050bb55341429a92ec2c53883730 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03a2050bb55341429a92ec2c53883730, 6213885d7cd04f5ea3c097bc97f2526b; no voters:
I20260926 06:22:44.394548 18214 raft_consensus.cc:2781] T 89b535d0d71d44208207d19532d4c232 P 03a2050bb55341429a92ec2c53883730 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:22:44.394626 18214 raft_consensus.cc:482] T 89b535d0d71d44208207d19532d4c232 P 03a2050bb55341429a92ec2c53883730 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:22:44.394652 18214 raft_consensus.cc:3037] T 89b535d0d71d44208207d19532d4c232 P 03a2050bb55341429a92ec2c53883730 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:22:44.394856 18222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:44.389002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:36317 (local address 127.4.202.129:60697)
0926 06:22:44.389370 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:44.389380 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:44.389467 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:44.389649 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:44.389649 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:44.389651 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:22:44.389725 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:44.389726 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:44.390831 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:44.390832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:44.392905 (+ 2073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:44.392908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:44.393673 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:44.393673 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:44.393746 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:44.393750 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:22:44.394717 (+ 967us) client_negotiation.cc:240] Negotiation successful
0926 06:22:44.394764 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":86,"threads_started":1}
I20260926 06:22:44.395326 17997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89b535d0d71d44208207d19532d4c232" candidate_uuid: "03a2050bb55341429a92ec2c53883730" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b9f9bc12465427b9af617c47dda742e" is_pre_election: true
I20260926 06:22:44.395466 17997 raft_consensus.cc:2454] T 89b535d0d71d44208207d19532d4c232 P 4b9f9bc12465427b9af617c47dda742e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03a2050bb55341429a92ec2c53883730 in term 0.
I20260926 06:22:44.395521 18214 raft_consensus.cc:504] T 89b535d0d71d44208207d19532d4c232 P 03a2050bb55341429a92ec2c53883730 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6213885d7cd04f5ea3c097bc97f2526b" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 42947 } } peers { permanent_uuid: "4b9f9bc12465427b9af617c47dda742e" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36317 } } peers { permanent_uuid: "03a2050bb55341429a92ec2c53883730" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 39293 } }
I20260926 06:22:44.395650 18214 leader_election.cc:288] T 89b535d0d71d44208207d19532d4c232 P 03a2050bb55341429a92ec2c53883730 [CANDIDATE]: Term 1 election: Requested vote from peers 6213885d7cd04f5ea3c097bc97f2526b (127.4.202.131:42947), 4b9f9bc12465427b9af617c47dda742e (127.4.202.130:36317)
I20260926 06:22:44.395807 18131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89b535d0d71d44208207d19532d4c232" candidate_uuid: "03a2050bb55341429a92ec2c53883730" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6213885d7cd04f5ea3c097bc97f2526b"
I20260926 06:22:44.395875 18131 raft_consensus.cc:3037] T 89b535d0d71d44208207d19532d4c232 P 6213885d7cd04f5ea3c097bc97f2526b [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:22:44.395936 17997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89b535d0d71d44208207d19532d4c232" candidate_uuid: "03a2050bb55341429a92ec2c53883730" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b9f9bc12465427b9af617c47dda742e"
I20260926 06:22:44.396008 17997 raft_consensus.cc:3037] T 89b535d0d71d44208207d19532d4c232 P 4b9f9bc12465427b9af617c47dda742e [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:22:44.396677 18131 raft_consensus.cc:2454] T 89b535d0d71d44208207d19532d4c232 P 6213885d7cd04f5ea3c097bc97f2526b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03a2050bb55341429a92ec2c53883730 in term 1.
I20260926 06:22:44.396847 17796 leader_election.cc:302] T 89b535d0d71d44208207d19532d4c232 P 03a2050bb55341429a92ec2c53883730 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03a2050bb55341429a92ec2c53883730, 6213885d7cd04f5ea3c097bc97f2526b; no voters:
I20260926 06:22:44.396871 17997 raft_consensus.cc:2454] T 89b535d0d71d44208207d19532d4c232 P 4b9f9bc12465427b9af617c47dda742e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03a2050bb55341429a92ec2c53883730 in term 1.
I20260926 06:22:44.396919 18214 raft_consensus.cc:2781] T 89b535d0d71d44208207d19532d4c232 P 03a2050bb55341429a92ec2c53883730 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:22:44.397045 18214 raft_consensus.cc:686] T 89b535d0d71d44208207d19532d4c232 P 03a2050bb55341429a92ec2c53883730 [term 1 LEADER]: Becoming Leader. State: Replica: 03a2050bb55341429a92ec2c53883730, State: Running, Role: LEADER
I20260926 06:22:44.397126 18214 consensus_queue.cc:237] T 89b535d0d71d44208207d19532d4c232 P 03a2050bb55341429a92ec2c53883730 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6213885d7cd04f5ea3c097bc97f2526b" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 42947 } } peers { permanent_uuid: "4b9f9bc12465427b9af617c47dda742e" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36317 } } peers { permanent_uuid: "03a2050bb55341429a92ec2c53883730" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 39293 } }
I20260926 06:22:44.397955 17711 catalog_manager.cc:5495] T 89b535d0d71d44208207d19532d4c232 P 03a2050bb55341429a92ec2c53883730 reported cstate change: term changed from 0 to 1, leader changed from <none> to 03a2050bb55341429a92ec2c53883730 (127.4.202.129). New cstate: current_term: 1 leader_uuid: "03a2050bb55341429a92ec2c53883730" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6213885d7cd04f5ea3c097bc97f2526b" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 42947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b9f9bc12465427b9af617c47dda742e" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03a2050bb55341429a92ec2c53883730" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 39293 } health_report { overall_health: HEALTHY } } }
I20260926 06:22:44.420707 18209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:44.417229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60038 (local address 127.4.202.129:39293)
0926 06:22:44.417285 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:44.417287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:44.417358 (+ 71us) server_negotiation.cc:407] Connection header received
0926 06:22:44.417451 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:44.417452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:44.417475 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:44.417558 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:22:44.417717 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:44.418443 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:44.419617 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:44.419915 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:44.419981 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:44.420179 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:22:44.420306 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:22:44.420383 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:44.420612 (+ 229us) server_negotiation.cc:299] Negotiation successful
0926 06:22:44.420631 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260926 06:22:44.422936 17997 raft_consensus.cc:1261] T 89b535d0d71d44208207d19532d4c232 P 4b9f9bc12465427b9af617c47dda742e [term 1 FOLLOWER]: Refusing update from remote peer 03a2050bb55341429a92ec2c53883730: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:22:44.423339 18223 consensus_queue.cc:1035] T 89b535d0d71d44208207d19532d4c232 P 03a2050bb55341429a92ec2c53883730 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b9f9bc12465427b9af617c47dda742e" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 36317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:22:44.425436 18131 raft_consensus.cc:1261] T 89b535d0d71d44208207d19532d4c232 P 6213885d7cd04f5ea3c097bc97f2526b [term 1 FOLLOWER]: Refusing update from remote peer 03a2050bb55341429a92ec2c53883730: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:22:44.425791 18223 consensus_queue.cc:1035] T 89b535d0d71d44208207d19532d4c232 P 03a2050bb55341429a92ec2c53883730 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6213885d7cd04f5ea3c097bc97f2526b" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 42947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:22:44.428658 18226 mvcc.cc:205] Tried to move back new op lower bound from 7333493819073122304 to 7333493818971164672. Current Snapshot: MvccSnapshot[applied={T|T < 7333493819073122304}]
I20260926 06:22:44.431280 18227 mvcc.cc:205] Tried to move back new op lower bound from 7333493819073122304 to 7333493818971164672. Current Snapshot: MvccSnapshot[applied={T|T < 7333493819073122304 or (T in {7333493819073122304})}]
I20260926 06:22:44.432474 17710 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:22:44.432569 17710 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:22:44.433240 17710 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eba7d6e554794125855840f041ebba50: Sending DeleteTablet for 3 replicas of tablet 89b535d0d71d44208207d19532d4c232
I20260926 06:22:44.433653 17965 tablet_service.cc:1514] Processing DeleteTablet for tablet 89b535d0d71d44208207d19532d4c232 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:22:44 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:53776
I20260926 06:22:44.433917 17843 tablet_service.cc:1514] Processing DeleteTablet for tablet 89b535d0d71d44208207d19532d4c232 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:22:44 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:50020
I20260926 06:22:44.434266 18111 tablet_service.cc:1514] Processing DeleteTablet for tablet 89b535d0d71d44208207d19532d4c232 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:22:44 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:54436
I20260926 06:22:44.434721 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 17749
I20260926 06:22:44.435328 18235 tablet_replica.cc:330] T 89b535d0d71d44208207d19532d4c232 P 4b9f9bc12465427b9af617c47dda742e: stopping tablet replica
I20260926 06:22:44.435464 18235 raft_consensus.cc:2229] T 89b535d0d71d44208207d19532d4c232 P 4b9f9bc12465427b9af617c47dda742e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:22:44.435611 18235 raft_consensus.cc:2258] T 89b535d0d71d44208207d19532d4c232 P 4b9f9bc12465427b9af617c47dda742e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:22:44.435813 18237 tablet_replica.cc:330] T 89b535d0d71d44208207d19532d4c232 P 6213885d7cd04f5ea3c097bc97f2526b: stopping tablet replica
I20260926 06:22:44.435941 18237 raft_consensus.cc:2229] T 89b535d0d71d44208207d19532d4c232 P 6213885d7cd04f5ea3c097bc97f2526b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:22:44.436064 18237 raft_consensus.cc:2258] T 89b535d0d71d44208207d19532d4c232 P 6213885d7cd04f5ea3c097bc97f2526b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:22:44.436518 18237 ts_tablet_manager.cc:1905] T 89b535d0d71d44208207d19532d4c232 P 6213885d7cd04f5ea3c097bc97f2526b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:22:44.436976 18235 ts_tablet_manager.cc:1905] T 89b535d0d71d44208207d19532d4c232 P 4b9f9bc12465427b9af617c47dda742e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:22:44.439891 18237 ts_tablet_manager.cc:1918] T 89b535d0d71d44208207d19532d4c232 P 6213885d7cd04f5ea3c097bc97f2526b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:22:44.440007 18237 log.cc:1196] T 89b535d0d71d44208207d19532d4c232 P 6213885d7cd04f5ea3c097bc97f2526b: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-2/wal/wals/89b535d0d71d44208207d19532d4c232
I20260926 06:22:44.440323 18237 ts_tablet_manager.cc:1939] T 89b535d0d71d44208207d19532d4c232 P 6213885d7cd04f5ea3c097bc97f2526b: Deleting consensus metadata
I20260926 06:22:44.440737 18235 ts_tablet_manager.cc:1918] T 89b535d0d71d44208207d19532d4c232 P 4b9f9bc12465427b9af617c47dda742e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:22:44.440861 18235 log.cc:1196] T 89b535d0d71d44208207d19532d4c232 P 4b9f9bc12465427b9af617c47dda742e: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790403591640726-4906-0/minicluster-data/ts-1/wal/wals/89b535d0d71d44208207d19532d4c232
I20260926 06:22:44.441164 18235 ts_tablet_manager.cc:1939] T 89b535d0d71d44208207d19532d4c232 P 4b9f9bc12465427b9af617c47dda742e: Deleting consensus metadata
I20260926 06:22:44.442708 17698 catalog_manager.cc:4841] TS 4b9f9bc12465427b9af617c47dda742e (127.4.202.130:36317): tablet 89b535d0d71d44208207d19532d4c232 (table test-table [id=38897f9ea6ad4e78b00b97300ca51f05]) successfully deleted
I20260926 06:22:44.442915 17695 catalog_manager.cc:4841] TS 6213885d7cd04f5ea3c097bc97f2526b (127.4.202.131:42947): tablet 89b535d0d71d44208207d19532d4c232 (table test-table [id=38897f9ea6ad4e78b00b97300ca51f05]) successfully deleted
W20260926 06:22:44.443463 17698 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.202.129:39293 (error 108)
I20260926 06:22:44.443996 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 17913
I20260926 06:22:44.444218 18204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:44.444107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:39293 (local address 127.0.0.1:60046)
0926 06:22:44.444151 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:44.444182 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.129:39293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260926 06:22:44.445515 17698 catalog_manager.cc:4584] TS 03a2050bb55341429a92ec2c53883730 (127.4.202.129:39293): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 89b535d0d71d44208207d19532d4c232: Network error: Client connection negotiation failed: client connection to 127.4.202.129:39293: connect: Connection refused (error 111)
I20260926 06:22:44.449434 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 18045
I20260926 06:22:44.456408 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 17678
I20260926 06:22:44.461203 4906 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260926 06:22:44.805240 4906 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260926 06:22:44.805346 4906 mini_ranger.cc:66] Stopping Ranger...
I20260926 06:22:45.149066 4906 mini_ranger.cc:68] Stopped Ranger
2026-09-26 06:22:45.149 UTC [16745] LOG: received smart shutdown request
2026-09-26 06:22:45.153 UTC [16745] LOG: background worker "logical replication launcher" (PID 16754) exited with exit code 1
2026-09-26 06:22:45.154 UTC [16749] LOG: shutting down
2026-09-26 06:22:45.320 UTC [16745] LOG: database system is shut down
2026-09-26T06:22:45Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (45288 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:22:45 dist-test-slave-pd0t krb5kdc[18247](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:22:45 dist-test-slave-pd0t krb5kdc[18247](info): set up 2 sockets
Sep 26 06:22:45 dist-test-slave-pd0t krb5kdc[18247](info): commencing operation
krb5kdc: starting...
W20260926 06:22:46.446341 4906 mini_kdc.cc:121] Time spent starting KDC: real 0.994s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:22:46 dist-test-slave-pd0t krb5kdc[18247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:22:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:22:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.202.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
I20260926 06:22:46.564647 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:43075
--webserver_interface=127.4.202.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:44001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:43075,127.4.202.189:39279,127.4.202.188:41027
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:22:46.641765 18260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:22:46.641938 18260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:22:46.641958 18260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:22:46.643040 18260 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:22:46.643076 18260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:22:46.643090 18260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:22:46.643102 18260 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:22:46.643152 18260 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:22:46.644572 18260 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:44001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:43075,127.4.202.189:39279,127.4.202.188:41027
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:43075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:22:46.644789 18260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:22:46.645013 18260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 06:22:46.649803 18260 server_base.cc:1034] running on GCE node
W20260926 06:22:46.649775 18265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:46.649775 18268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:46.649823 18266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:22:46.650204 18260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:22:46.650530 18260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:22:46.651722 18260 hybrid_clock.cc:648] HybridClock initialized: now 1790403766651698 us; error 40 us; skew 500 ppm
Sep 26 06:22:46 dist-test-slave-pd0t krb5kdc[18247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403766, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:22:46.654045 18260 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:22:46.654456 18260 webserver.cc:460] Webserver started at http://127.4.202.190:46651/ using document root <none> and password file <none>
I20260926 06:22:46.654708 18260 fs_manager.cc:359] Metadata directory not provided
I20260926 06:22:46.654771 18260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:22:46.654896 18260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:22:46.655860 18260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "dcab76b12dbd45d7b6c052b0b49e106c"
format_stamp: "Formatted at 2026-09-26 06:22:46 on dist-test-slave-pd0t"
server_key: "11d1b90b27df62b520856b5f57387f39"
server_key_iv: "5f2df059ef619dcc3facd4a07bccf933"
server_key_version: "encryptionkey@0"
I20260926 06:22:46.656193 18260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "dcab76b12dbd45d7b6c052b0b49e106c"
format_stamp: "Formatted at 2026-09-26 06:22:46 on dist-test-slave-pd0t"
server_key: "11d1b90b27df62b520856b5f57387f39"
server_key_iv: "5f2df059ef619dcc3facd4a07bccf933"
server_key_version: "encryptionkey@0"
I20260926 06:22:46.657428 18260 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:22:46.658142 18275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:46.658331 18260 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:22:46.658428 18260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "dcab76b12dbd45d7b6c052b0b49e106c"
format_stamp: "Formatted at 2026-09-26 06:22:46 on dist-test-slave-pd0t"
server_key: "11d1b90b27df62b520856b5f57387f39"
server_key_iv: "5f2df059ef619dcc3facd4a07bccf933"
server_key_version: "encryptionkey@0"
I20260926 06:22:46.658507 18260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:22:46.676156 18260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:22:46.677873 18260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:22:46.678021 18260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:22:46.681845 18260 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:43075
I20260926 06:22:46.681881 18327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:43075 every 8 connection(s)
I20260926 06:22:46.682216 18260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:22:46.683104 18328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:22:46.684571 18328 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43075 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:22:46.689090 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 18260
I20260926 06:22:46.689186 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:22:46.689353 4906 external_mini_cluster.cc:1428] Setting key 3bfb93210df5489f0aaf41757d125513
Sep 26 06:22:46 dist-test-slave-pd0t krb5kdc[18247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403766, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:22:46.694003 18331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:46.685172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56996 (local address 127.4.202.190:43075)
0926 06:22:46.685922 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:46.685925 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:46.685935 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:22:46.685988 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:46.685990 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:46.686004 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:46.686065 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:46.686652 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.687450 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.688296 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.688608 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.688681 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:46.690880 (+ 2199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:46.690887 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:46.690888 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:46.690894 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:46.692491 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:46.692931 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:46.692933 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:46.692933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:46.693008 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:46.693290 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:46.693291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:46.693291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:46.693479 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:46.693539 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:46.693772 (+ 233us) server_negotiation.cc:299] Negotiation successful
0926 06:22:46.693863 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":731,"thread_start_us":62,"threads_started":1}
I20260926 06:22:46.694165 18330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:46.685130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:43075 (local address 127.0.0.1:56996)
0926 06:22:46.685340 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:46.685373 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:46.685799 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:46.686129 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:46.686131 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:46.686369 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:22:46.686511 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:46.686515 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.687537 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.687538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:46.688203 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:46.688205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.688686 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.688687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:46.688743 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:46.689308 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:22:46.689321 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:22:46.690799 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:22:46.692577 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:46.692584 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:46.692586 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:46.692846 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:46.693081 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:46.693082 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:46.693082 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:46.693191 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:46.693550 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:22:46.693556 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:22:46.693635 (+ 79us) client_negotiation.cc:769] Sending connection context
0926 06:22:46.693745 (+ 110us) client_negotiation.cc:240] Negotiation successful
0926 06:22:46.693969 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":59,"thread_start_us":89,"threads_started":1}
I20260926 06:22:46.694900 18328 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.202.189" port: 39279 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:22:46.695217 18330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:46.695099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:39279 (local address 127.0.0.1:37222)
0926 06:22:46.695145 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:46.695191 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.189:39279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260926 06:22:46.695364 18276 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.202.189:39279: connect: Connection refused (error 111)
I20260926 06:22:46.695783 18330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:46.695677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:39279 (local address 127.0.0.1:37226)
0926 06:22:46.695726 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:46.695756 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.189:39279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260926 06:22:46.695894 18328 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.202.189:39279: Network error: Client connection negotiation failed: client connection to 127.4.202.189:39279: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.4.202.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.189.keytab.
Entry for principal kudu/127.4.202.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.189.keytab.
I20260926 06:22:46.713462 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.189:39279
--webserver_interface=127.4.202.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.189.keytab
--principal=kudu/127.4.202.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:44001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:43075,127.4.202.189:39279,127.4.202.188:41027
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:22:46.745116 18328 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.202.189" port: 39279 } attempt: 1
I20260926 06:22:46.745621 18330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:46.745484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:39279 (local address 127.0.0.1:37236)
0926 06:22:46.745552 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:46.745592 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.189:39279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260926 06:22:46.745987 18330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:46.745893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:39279 (local address 127.0.0.1:37240)
0926 06:22:46.745935 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:46.745962 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.189:39279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260926 06:22:46.746098 18328 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.202.189:39279: Network error: Client connection negotiation failed: client connection to 127.4.202.189:39279: connect: Connection refused (error 111)
W20260926 06:22:46.791749 18335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:22:46.791932 18335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:22:46.791952 18335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:22:46.793113 18335 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:22:46.793155 18335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:22:46.793169 18335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:22:46.793208 18335 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:22:46.793236 18335 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:22:46.794695 18335 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:44001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:43075,127.4.202.189:39279,127.4.202.188:41027
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.189.keytab
--principal=kudu/127.4.202.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.189:39279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.202.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:22:46.794926 18335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:22:46.795137 18335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:22:46.799674 18343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:46.799713 18341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:46.799674 18340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:22:46.799911 18335 server_base.cc:1034] running on GCE node
I20260926 06:22:46.800190 18335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:22:46.800457 18335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:22:46.801623 18335 hybrid_clock.cc:648] HybridClock initialized: now 1790403766801605 us; error 36 us; skew 500 ppm
Sep 26 06:22:46 dist-test-slave-pd0t krb5kdc[18247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403766, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:22:46.804106 18335 init.cc:377] Logged in from keytab as kudu/127.4.202.189@KRBTEST.COM (short username kudu)
I20260926 06:22:46.804538 18335 webserver.cc:460] Webserver started at http://127.4.202.189:44425/ using document root <none> and password file <none>
I20260926 06:22:46.804754 18335 fs_manager.cc:359] Metadata directory not provided
I20260926 06:22:46.804826 18335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:22:46.804965 18335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:22:46.805919 18335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-1/data/instance:
uuid: "9567db560fc248129de3bfa164c49794"
format_stamp: "Formatted at 2026-09-26 06:22:46 on dist-test-slave-pd0t"
server_key: "7b6b95f017e51b07fc1fb6d1a96f1fce"
server_key_iv: "30dc305fe479db21844560a2053600e0"
server_key_version: "encryptionkey@0"
I20260926 06:22:46.806252 18335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-1/wal/instance:
uuid: "9567db560fc248129de3bfa164c49794"
format_stamp: "Formatted at 2026-09-26 06:22:46 on dist-test-slave-pd0t"
server_key: "7b6b95f017e51b07fc1fb6d1a96f1fce"
server_key_iv: "30dc305fe479db21844560a2053600e0"
server_key_version: "encryptionkey@0"
I20260926 06:22:46.807591 18335 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:22:46.808301 18350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:46.808507 18335 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:46.808604 18335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-1/wal
uuid: "9567db560fc248129de3bfa164c49794"
format_stamp: "Formatted at 2026-09-26 06:22:46 on dist-test-slave-pd0t"
server_key: "7b6b95f017e51b07fc1fb6d1a96f1fce"
server_key_iv: "30dc305fe479db21844560a2053600e0"
server_key_version: "encryptionkey@0"
I20260926 06:22:46.808676 18335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:22:46.814268 18328 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.202.189" port: 39279 } attempt: 2
I20260926 06:22:46.814657 18330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:46.814511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:39279 (local address 127.0.0.1:37250)
0926 06:22:46.814570 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:46.814624 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.189:39279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260926 06:22:46.814988 18330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:46.814909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:39279 (local address 127.0.0.1:37256)
0926 06:22:46.814950 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:46.814973 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.189:39279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260926 06:22:46.815100 18328 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.202.189:39279: Network error: Client connection negotiation failed: client connection to 127.4.202.189:39279: connect: Connection refused (error 111)
I20260926 06:22:46.819350 18335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:22:46.821053 18335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:22:46.821198 18335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:22:46.825049 18335 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.189:39279
I20260926 06:22:46.825114 18402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.189:39279 every 8 connection(s)
I20260926 06:22:46.825405 18335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-1/data/info.pb
I20260926 06:22:46.826150 18403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:22:46.827392 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 18335
I20260926 06:22:46.827481 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-1/wal/instance
I20260926 06:22:46.827658 4906 external_mini_cluster.cc:1428] Setting key 5141bfda3dcf312dd6359cfb834535e4
I20260926 06:22:46.827875 18403 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43075 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 06:22:46 dist-test-slave-pd0t krb5kdc[18247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403766, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.189@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.202.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.188@KRBTEST.COM" created.
I20260926 06:22:46.837699 18331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:46.828525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56998 (local address 127.4.202.190:43075)
0926 06:22:46.828593 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:46.828595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:46.828848 (+ 253us) server_negotiation.cc:407] Connection header received
0926 06:22:46.829316 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:46.829318 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:46.829339 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:46.829408 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:46.830033 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.830499 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.832021 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.832218 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.832319 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:46.834560 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:46.834562 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:46.834562 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:46.834569 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:46.836268 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:46.836683 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:46.836684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:46.836685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:46.836747 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:46.837048 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:46.837049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:46.837049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:46.837152 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:46.837205 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:46.837545 (+ 340us) server_negotiation.cc:299] Negotiation successful
0926 06:22:46.837633 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 06:22:46.837875 18405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:46.828511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:43075 (local address 127.0.0.1:56998)
0926 06:22:46.828695 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:46.828741 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:46.829228 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:46.829467 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:46.829470 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:46.829683 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:22:46.829947 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:46.829952 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.830612 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.830613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:46.831918 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:46.831922 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.832337 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.832339 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:46.832509 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:46.832981 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:22:46.832991 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:22:46.834465 (+ 1474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:22:46.836372 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:46.836379 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:46.836382 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:46.836598 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:46.836879 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:46.836879 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:46.836880 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:46.836981 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:46.837236 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:22:46.837242 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:22:46.837359 (+ 117us) client_negotiation.cc:769] Sending connection context
0926 06:22:46.837497 (+ 138us) client_negotiation.cc:240] Negotiation successful
0926 06:22:46.837667 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":88,"threads_started":1}
I20260926 06:22:46.839118 18403 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.202.189" port: 39279 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 06:22:46 dist-test-slave-pd0t krb5kdc[18247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403766, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.189@KRBTEST.COM for kudu/127.4.202.189@KRBTEST.COM
I20260926 06:22:46.845961 18405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:46.839361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:39279 (local address 127.0.0.1:37272)
0926 06:22:46.839407 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:46.839417 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:46.839453 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:46.839705 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:46.839706 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:46.839845 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:22:46.839882 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:46.839883 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.840706 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.840707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:46.841172 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:46.841173 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.841623 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.841623 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:46.841684 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:46.842086 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:22:46.842089 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:22:46.843423 (+ 1334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:22:46.844971 (+ 1548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:46.844972 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:46.844972 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:46.845122 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:46.845401 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:46.845402 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:46.845402 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:46.845451 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:46.845713 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:22:46.845713 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:22:46.845748 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 06:22:46.845831 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 06:22:46.845904 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260926 06:22:46.846053 18408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:46.839429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37272 (local address 127.4.202.189:39279)
0926 06:22:46.839585 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:46.839588 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:46.839603 (+ 15us) server_negotiation.cc:407] Connection header received
0926 06:22:46.839619 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:46.839620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:46.839630 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:46.839705 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:46.839973 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.840612 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.841261 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.841551 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.841625 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:46.843501 (+ 1876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:46.843507 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:46.843507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:46.843513 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:46.844895 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:46.845249 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:46.845252 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:46.845252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:46.845342 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:46.845505 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:46.845505 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:46.845505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:46.845645 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:46.845703 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:46.845865 (+ 162us) server_negotiation.cc:299] Negotiation successful
0926 06:22:46.845950 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":133,"thread_start_us":87,"threads_started":1}
I20260926 06:22:46.846747 18403 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.202.188" port: 41027 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:22:46.847066 18405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:46.846965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:41027 (local address 127.0.0.1:42030)
0926 06:22:46.847002 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:46.847045 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.188:41027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260926 06:22:46.847216 18354 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.202.188:41027: connect: Connection refused (error 111)
I20260926 06:22:46.847627 18405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:46.847535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:41027 (local address 127.0.0.1:42034)
0926 06:22:46.847568 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:46.847612 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.188:41027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260926 06:22:46.847734 18403 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.202.188:41027: Network error: Client connection negotiation failed: client connection to 127.4.202.188:41027: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.188.keytab.
Entry for principal kudu/127.4.202.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.188.keytab.
I20260926 06:22:46.853010 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.188:41027
--webserver_interface=127.4.202.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.188.keytab
--principal=kudu/127.4.202.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:44001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:43075,127.4.202.189:39279,127.4.202.188:41027
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:22:46.897008 18403 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.202.188" port: 41027 } attempt: 1
I20260926 06:22:46.897497 18405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:46.897366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:41027 (local address 127.0.0.1:42038)
0926 06:22:46.897432 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:46.897469 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.188:41027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260926 06:22:46.897840 18405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:46.897743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:41027 (local address 127.0.0.1:42054)
0926 06:22:46.897792 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:46.897817 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.188:41027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260926 06:22:46.897924 18403 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.202.188:41027: Network error: Client connection negotiation failed: client connection to 127.4.202.188:41027: connect: Connection refused (error 111)
I20260926 06:22:46.906236 18328 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.202.189" port: 39279 } attempt: 3
Sep 26 06:22:46 dist-test-slave-pd0t krb5kdc[18247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403766, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for kudu/127.4.202.189@KRBTEST.COM
I20260926 06:22:46.913318 18330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:46.906503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:39279 (local address 127.0.0.1:37278)
0926 06:22:46.906557 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:46.906588 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:46.906653 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:46.906824 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:46.906825 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:46.906998 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:22:46.907059 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:46.907060 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.907741 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.907742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:46.908291 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:46.908293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.908675 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.908676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:46.908753 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:46.909197 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:22:46.909202 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:22:46.910452 (+ 1250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:22:46.912341 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:46.912341 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:46.912342 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:46.912507 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:46.912733 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:46.912734 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:46.912734 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:46.912785 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:46.913066 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:22:46.913067 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:22:46.913105 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 06:22:46.913155 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 06:22:46.913224 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 06:22:46.913401 18408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:46.906557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37278 (local address 127.4.202.189:39279)
0926 06:22:46.906626 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:46.906628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:46.906640 (+ 12us) server_negotiation.cc:407] Connection header received
0926 06:22:46.906729 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:46.906730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:46.906739 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:46.906819 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:46.907141 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.907653 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.908387 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.908591 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.908674 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:46.910538 (+ 1864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:46.910540 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:46.910540 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:46.910547 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:46.912190 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:46.912583 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:46.912584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:46.912584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:46.912663 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:46.912850 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:46.912851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:46.912851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:46.912999 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:46.913077 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:46.913184 (+ 107us) server_negotiation.cc:299] Negotiation successful
0926 06:22:46.913272 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 06:22:46.913757 18328 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.202.188" port: 41027 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:22:46.914130 18330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:46.914008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:41027 (local address 127.0.0.1:42070)
0926 06:22:46.914065 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:46.914101 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.188:41027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260926 06:22:46.914508 18330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:46.914399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:41027 (local address 127.0.0.1:42082)
0926 06:22:46.914446 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:46.914486 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.188:41027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260926 06:22:46.914618 18328 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.202.188:41027: Network error: Client connection negotiation failed: client connection to 127.4.202.188:41027: connect: Connection refused (error 111)
W20260926 06:22:46.931165 18410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:22:46.931350 18410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:22:46.931371 18410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:22:46.932510 18410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:22:46.932548 18410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:22:46.932561 18410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:22:46.932600 18410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:22:46.932624 18410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:22:46.934062 18410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:44001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:43075,127.4.202.189:39279,127.4.202.188:41027
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.188.keytab
--principal=kudu/127.4.202.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.188:41027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.202.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:22:46.934314 18410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:22:46.934563 18410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:22:46.939265 18415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:22:46.939337 18410 server_base.cc:1034] running on GCE node
W20260926 06:22:46.939275 18416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:46.939302 18418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:22:46.939774 18410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:22:46.940047 18410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:22:46.941172 18410 hybrid_clock.cc:648] HybridClock initialized: now 1790403766941152 us; error 33 us; skew 500 ppm
Sep 26 06:22:46 dist-test-slave-pd0t krb5kdc[18247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403766, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:22:46.943485 18410 init.cc:377] Logged in from keytab as kudu/127.4.202.188@KRBTEST.COM (short username kudu)
I20260926 06:22:46.943828 18410 webserver.cc:460] Webserver started at http://127.4.202.188:42131/ using document root <none> and password file <none>
I20260926 06:22:46.943990 18410 fs_manager.cc:359] Metadata directory not provided
I20260926 06:22:46.944028 18410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:22:46.944110 18410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:22:46.945034 18410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-2/data/instance:
uuid: "cf1268b6fba945eba3eb5b1f8f4a292f"
format_stamp: "Formatted at 2026-09-26 06:22:46 on dist-test-slave-pd0t"
server_key: "9516ebaee937edad69334fafe2ac95b1"
server_key_iv: "3169def59dca3e062352022bb74a6566"
server_key_version: "encryptionkey@0"
I20260926 06:22:46.945345 18410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-2/wal/instance:
uuid: "cf1268b6fba945eba3eb5b1f8f4a292f"
format_stamp: "Formatted at 2026-09-26 06:22:46 on dist-test-slave-pd0t"
server_key: "9516ebaee937edad69334fafe2ac95b1"
server_key_iv: "3169def59dca3e062352022bb74a6566"
server_key_version: "encryptionkey@0"
I20260926 06:22:46.945748 18328 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.202.188" port: 41027 } attempt: 1
I20260926 06:22:46.946080 18330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:46.945971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:41027 (local address 127.0.0.1:42084)
0926 06:22:46.946026 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:46.946057 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.188:41027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260926 06:22:46.946521 18330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:46.946365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:41027 (local address 127.0.0.1:42096)
0926 06:22:46.946453 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:46.946481 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.188:41027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260926 06:22:46.946636 18328 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.202.188:41027: Network error: Client connection negotiation failed: client connection to 127.4.202.188:41027: connect: Connection refused (error 111)
I20260926 06:22:46.946768 18410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:22:46.947579 18425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:46.947759 18410 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:22:46.947844 18410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-2/wal
uuid: "cf1268b6fba945eba3eb5b1f8f4a292f"
format_stamp: "Formatted at 2026-09-26 06:22:46 on dist-test-slave-pd0t"
server_key: "9516ebaee937edad69334fafe2ac95b1"
server_key_iv: "3169def59dca3e062352022bb74a6566"
server_key_version: "encryptionkey@0"
I20260926 06:22:46.947894 18410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:22:46.959447 18410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:22:46.961262 18410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:22:46.961403 18410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:22:46.964874 18410 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.188:41027
I20260926 06:22:46.964936 18477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.188:41027 every 8 connection(s)
I20260926 06:22:46.965214 18410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-2/data/info.pb
I20260926 06:22:46.966061 18403 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.202.188" port: 41027 } attempt: 2
I20260926 06:22:46.966118 18478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:22:46.966820 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 18410
I20260926 06:22:46.966908 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790403591640726-4906-0/minicluster-data/master-2/wal/instance
I20260926 06:22:46.967092 4906 external_mini_cluster.cc:1428] Setting key bf3cc184c31dc78743196585c886bf9b
I20260926 06:22:46.968344 18478 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43075 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 06:22:46 dist-test-slave-pd0t krb5kdc[18247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403766, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.189@KRBTEST.COM for kudu/127.4.202.188@KRBTEST.COM
Sep 26 06:22:46 dist-test-slave-pd0t krb5kdc[18247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.188@KRBTEST.COM
Sep 26 06:22:46 dist-test-slave-pd0t krb5kdc[18247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403766, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.188@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:22:46.977579 18405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:46.966361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:41027 (local address 127.0.0.1:42112)
0926 06:22:46.966417 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:46.966428 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:46.966470 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:46.967933 (+ 1463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:46.967934 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:46.968113 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:22:46.968155 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:46.968157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.969572 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.969574 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:46.970379 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:46.970383 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.971752 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.971753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:46.971812 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:46.972479 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:22:46.972483 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:22:46.973851 (+ 1368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:22:46.976256 (+ 2405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:46.976258 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:46.976258 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:46.976489 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:46.976771 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:46.976772 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:46.976773 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:46.976853 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:46.977225 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:22:46.977226 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:22:46.977276 (+ 50us) client_negotiation.cc:769] Sending connection context
0926 06:22:46.977359 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 06:22:46.977459 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260926 06:22:46.977777 18480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:46.966519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42112 (local address 127.4.202.188:41027)
0926 06:22:46.966992 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:46.967001 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:46.967032 (+ 31us) server_negotiation.cc:407] Connection header received
0926 06:22:46.967739 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:46.967749 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:46.967804 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:46.967946 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:46.968254 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.969478 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.971307 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.971668 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.971782 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:46.973957 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:46.973976 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:46.973976 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:46.973982 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:46.976138 (+ 2156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:46.976581 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:46.976584 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:46.976584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:46.976675 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:46.976904 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:46.976905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:46.976906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:46.977156 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:46.977228 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:46.977433 (+ 205us) server_negotiation.cc:299] Negotiation successful
0926 06:22:46.977554 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":139,"threads_started":1}
I20260926 06:22:46.978137 18483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:46.968891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:43075 (local address 127.0.0.1:57010)
0926 06:22:46.969608 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:46.969650 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:46.969723 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:46.969864 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:46.969866 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:46.970103 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:22:46.970156 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:46.970157 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.971033 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.971034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:46.971826 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:46.971828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.972341 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.972342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:46.972420 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:46.972872 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:22:46.972882 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:22:46.975187 (+ 2305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:22:46.976833 (+ 1646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:46.976839 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:46.976841 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:46.977105 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:46.977292 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:46.977292 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:46.977293 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:46.977361 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:46.977765 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:22:46.977771 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:22:46.977821 (+ 50us) client_negotiation.cc:769] Sending connection context
0926 06:22:46.977919 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 06:22:46.978019 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"thread_start_us":78,"threads_started":1}
I20260926 06:22:46.978153 18331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:46.969097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57010 (local address 127.4.202.190:43075)
0926 06:22:46.969135 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:46.969137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:46.969718 (+ 581us) server_negotiation.cc:407] Connection header received
0926 06:22:46.969779 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:46.969780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:46.969791 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:46.969839 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:46.970245 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.970955 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.971935 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.972228 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.972314 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:46.975291 (+ 2977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:46.975292 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:46.975293 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:46.975300 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:46.976750 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:46.977180 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:46.977182 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:46.977182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:46.977238 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:46.977442 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:46.977443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:46.977443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:46.977570 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:46.977613 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:46.977946 (+ 333us) server_negotiation.cc:299] Negotiation successful
0926 06:22:46.978029 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 06:22:46.979024 18478 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.202.189" port: 39279 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:22:46.979750 18403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9567db560fc248129de3bfa164c49794: Bootstrap starting.
I20260926 06:22:46.980760 18403 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9567db560fc248129de3bfa164c49794: Neither blocks nor log segments found. Creating new log.
I20260926 06:22:46.981060 18403 log.cc:824] T 00000000000000000000000000000000 P 9567db560fc248129de3bfa164c49794: Log is configured to *not* fsync() on all Append() calls
I20260926 06:22:46.981355 18481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:46.967392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42116 (local address 127.4.202.188:41027)
0926 06:22:46.967527 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:46.967529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:46.967687 (+ 158us) server_negotiation.cc:407] Connection header received
0926 06:22:46.968760 (+ 1073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:46.968761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:46.968771 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:46.968856 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:46.969571 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.970388 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.971306 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.971670 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.972018 (+ 348us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:46.974568 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:46.974569 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:46.974569 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:46.974594 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:46.976919 (+ 2325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:46.977324 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:46.977325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:46.977326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:46.977380 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:46.977581 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:46.977582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:46.977582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:46.977709 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:46.978298 (+ 589us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:46.978334 (+ 36us) server_negotiation.cc:299] Negotiation successful
0926 06:22:46.978389 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":78,"threads_started":1}
I20260926 06:22:46.981887 18403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9567db560fc248129de3bfa164c49794: No bootstrap required, opened a new log
Sep 26 06:22:46 dist-test-slave-pd0t krb5kdc[18247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403766, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.188@KRBTEST.COM for kudu/127.4.202.189@KRBTEST.COM
I20260926 06:22:46.983954 18403 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9567db560fc248129de3bfa164c49794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcab76b12dbd45d7b6c052b0b49e106c" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43075 } } peers { permanent_uuid: "9567db560fc248129de3bfa164c49794" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 39279 } } peers { permanent_uuid: "cf1268b6fba945eba3eb5b1f8f4a292f" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 41027 } }
I20260926 06:22:46.984100 18403 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9567db560fc248129de3bfa164c49794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:22:46.984153 18403 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9567db560fc248129de3bfa164c49794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9567db560fc248129de3bfa164c49794, State: Initialized, Role: FOLLOWER
I20260926 06:22:46.984297 18403 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9567db560fc248129de3bfa164c49794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcab76b12dbd45d7b6c052b0b49e106c" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43075 } } peers { permanent_uuid: "9567db560fc248129de3bfa164c49794" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 39279 } } peers { permanent_uuid: "cf1268b6fba945eba3eb5b1f8f4a292f" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 41027 } }
I20260926 06:22:46.984629 18403 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9567db560fc248129de3bfa164c49794 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:22:46.986056 18483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:46.979265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:39279 (local address 127.0.0.1:37284)
0926 06:22:46.979376 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:46.979386 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:46.979437 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:46.979541 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:46.979541 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:46.979671 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:22:46.979718 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:46.979720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.980352 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.980353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:46.981092 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:46.981095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.981513 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.981514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:46.981577 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:46.981959 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:22:46.981963 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:22:46.983252 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:22:46.984929 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:46.984930 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:46.984930 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:46.985083 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:46.985311 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:46.985311 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:46.985312 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:46.985377 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:46.985688 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:22:46.985689 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:22:46.985740 (+ 51us) client_negotiation.cc:769] Sending connection context
0926 06:22:46.985834 (+ 94us) client_negotiation.cc:240] Negotiation successful
0926 06:22:46.985938 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260926 06:22:46.986029 18408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:46.979333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37284 (local address 127.4.202.189:39279)
0926 06:22:46.979378 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:46.979380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:46.979424 (+ 44us) server_negotiation.cc:407] Connection header received
0926 06:22:46.979490 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:46.979490 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:46.979497 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:46.979536 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:46.979785 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.980267 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.981176 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.981379 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.981455 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:46.983339 (+ 1884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:46.983340 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:46.983341 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:46.983348 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:46.984840 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:46.985176 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:46.985178 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:46.985178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:46.985233 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:46.985424 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:46.985425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:46.985425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:46.985591 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:46.985655 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:46.985867 (+ 212us) server_negotiation.cc:299] Negotiation successful
0926 06:22:46.985948 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 06:22:46.984596 18485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9567db560fc248129de3bfa164c49794 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcab76b12dbd45d7b6c052b0b49e106c" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43075 } } peers { permanent_uuid: "9567db560fc248129de3bfa164c49794" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 39279 } } peers { permanent_uuid: "cf1268b6fba945eba3eb5b1f8f4a292f" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 41027 } } }
I20260926 06:22:46.986434 18485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9567db560fc248129de3bfa164c49794 [sys.catalog]: This master's current role is: FOLLOWER
W20260926 06:22:46.986670 18496 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9567db560fc248129de3bfa164c49794: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:22:46.986744 18496 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:22:46.987098 18478 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.202.188" port: 41027 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:22:46.986783 18335 master_runner.cc:186] Error getting master registration for 127.4.202.190:43075: OK, instance_id { permanent_uuid: "dcab76b12dbd45d7b6c052b0b49e106c" instance_seqno: 1790403766680821 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 06:22:46.988992 18335 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Sep 26 06:22:46 dist-test-slave-pd0t krb5kdc[18247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
Sep 26 06:22:46 dist-test-slave-pd0t krb5kdc[18247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403766, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.188@KRBTEST.COM for kudu/127.4.202.188@KRBTEST.COM
I20260926 06:22:46.993664 18483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:46.987318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:41027 (local address 127.0.0.1:42120)
0926 06:22:46.987369 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:46.987378 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:46.987414 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:46.987557 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:46.987557 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:46.987684 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:22:46.987709 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:46.987711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.988548 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.988549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:46.989210 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:46.989212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.989578 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.989578 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:46.989625 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:46.989994 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:22:46.989998 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:22:46.991127 (+ 1129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:22:46.992633 (+ 1506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:46.992634 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:46.992634 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:46.992798 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:46.993001 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:46.993001 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:46.993002 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:46.993054 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:46.993323 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:22:46.993323 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:22:46.993360 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 06:22:46.993448 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 06:22:46.993541 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260926 06:22:46.993805 18331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:46.981888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57026 (local address 127.4.202.190:43075)
0926 06:22:46.981969 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:46.981971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:46.982709 (+ 738us) server_negotiation.cc:407] Connection header received
0926 06:22:46.982777 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:46.982777 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:46.982787 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:46.982853 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:46.983407 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.984585 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.985547 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.985856 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.985917 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:46.990896 (+ 4979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:46.990897 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:46.990898 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:46.990905 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:46.992538 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:46.992937 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:46.992938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:46.992939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:46.992991 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:46.993189 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:46.993190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:46.993190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:46.993303 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:46.993437 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:46.993645 (+ 208us) server_negotiation.cc:299] Negotiation successful
0926 06:22:46.993720 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 06:22:46.993718 18481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:46.987374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42120 (local address 127.4.202.188:41027)
0926 06:22:46.987408 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:46.987410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:46.987417 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:22:46.987472 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:46.987473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:46.987481 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:46.987528 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:46.987783 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.988466 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.989291 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.989485 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.989576 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:46.991198 (+ 1622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:46.991199 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:46.991199 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:46.991207 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:46.992530 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:46.992881 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:46.992882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:46.992882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:46.992929 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:46.993125 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:46.993126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:46.993126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:46.993250 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:46.993326 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:46.993473 (+ 147us) server_negotiation.cc:299] Negotiation successful
0926 06:22:46.993591 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":20}
I20260926 06:22:46.995381 18478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f: Bootstrap starting.
I20260926 06:22:46.995990 18478 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f: Neither blocks nor log segments found. Creating new log.
I20260926 06:22:46.996289 18478 log.cc:824] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f: Log is configured to *not* fsync() on all Append() calls
I20260926 06:22:46.996971 18478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f: No bootstrap required, opened a new log
I20260926 06:22:46.998474 18478 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcab76b12dbd45d7b6c052b0b49e106c" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43075 } } peers { permanent_uuid: "9567db560fc248129de3bfa164c49794" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 39279 } } peers { permanent_uuid: "cf1268b6fba945eba3eb5b1f8f4a292f" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 41027 } }
I20260926 06:22:46.998672 18478 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:22:46.998746 18478 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf1268b6fba945eba3eb5b1f8f4a292f, State: Initialized, Role: FOLLOWER
I20260926 06:22:46.998874 18478 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcab76b12dbd45d7b6c052b0b49e106c" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43075 } } peers { permanent_uuid: "9567db560fc248129de3bfa164c49794" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 39279 } } peers { permanent_uuid: "cf1268b6fba945eba3eb5b1f8f4a292f" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 41027 } }
Sep 26 06:22:46 dist-test-slave-pd0t krb5kdc[18247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.189@KRBTEST.COM
I20260926 06:22:46.999166 18478 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:22:46.999136 18499 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcab76b12dbd45d7b6c052b0b49e106c" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43075 } } peers { permanent_uuid: "9567db560fc248129de3bfa164c49794" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 39279 } } peers { permanent_uuid: "cf1268b6fba945eba3eb5b1f8f4a292f" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 41027 } } }
I20260926 06:22:46.999946 18499 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f [sys.catalog]: This master's current role is: FOLLOWER
W20260926 06:22:47.002175 18510 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:22:47.002270 18510 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:22:47.002787 18408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:46.994512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37290 (local address 127.4.202.189:39279)
0926 06:22:46.994544 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:46.994545 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:46.994553 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:22:46.994645 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:46.994646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:46.994654 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:46.994698 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:46.995241 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.995735 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.996646 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:46.996940 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:46.996982 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:46.999635 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:46.999637 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:46.999637 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:46.999643 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:47.001221 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:47.001633 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:47.001634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:47.001634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:47.001695 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:47.001941 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:47.001942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:47.001942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:47.002062 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:47.002212 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:47.002632 (+ 420us) server_negotiation.cc:299] Negotiation successful
0926 06:22:47.002701 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260926 06:22:47.001870 18410 master_runner.cc:186] Error getting master registration for 127.4.202.190:43075: OK, instance_id { permanent_uuid: "dcab76b12dbd45d7b6c052b0b49e106c" instance_seqno: 1790403766680821 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 06:22:47.004127 18410 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 06:22:47.004580 4906 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:22:47.015628 18481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:47.006350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42128 (local address 127.4.202.188:41027)
0926 06:22:47.006382 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:47.006384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:47.006893 (+ 509us) server_negotiation.cc:407] Connection header received
0926 06:22:47.006945 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:47.006946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:47.006956 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:47.007008 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:47.007704 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:47.008432 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:47.009459 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:47.009745 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:47.009783 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:47.011610 (+ 1827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:47.011611 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:47.011612 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:47.011619 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:47.013484 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:47.014686 (+ 1202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:47.014687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:47.014687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:47.014749 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:47.014980 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:47.014981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:47.014982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:47.015114 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:47.015217 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:47.015441 (+ 224us) server_negotiation.cc:299] Negotiation successful
0926 06:22:47.015527 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 06:22:47.017828 18331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:47.006343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57034 (local address 127.4.202.190:43075)
0926 06:22:47.006383 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:47.006385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:47.006763 (+ 378us) server_negotiation.cc:407] Connection header received
0926 06:22:47.006810 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:47.006811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:47.006830 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:47.006888 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:47.010702 (+ 3814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:47.011374 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:47.012357 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:47.013108 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:47.013175 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:47.014568 (+ 1393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:47.014569 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:47.014569 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:47.014595 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:47.016331 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:47.016979 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:47.016980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:47.016980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:47.017042 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:47.017244 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:47.017245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:47.017245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:47.017379 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:47.017445 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:47.017661 (+ 216us) server_negotiation.cc:299] Negotiation successful
0926 06:22:47.017740 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 06:22:47.018116 18408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:47.006247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37300 (local address 127.4.202.189:39279)
0926 06:22:47.006296 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:47.006298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:47.007026 (+ 728us) server_negotiation.cc:407] Connection header received
0926 06:22:47.007133 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:47.007134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:47.007145 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:47.007173 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:47.007820 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:47.008525 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:47.009459 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:47.009745 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:47.009783 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:47.011611 (+ 1828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:47.011612 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:47.011612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:47.011619 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:47.014219 (+ 2600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:47.017284 (+ 3065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:47.017285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:47.017285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:47.017333 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:47.017540 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:47.017540 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:47.017541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:47.017666 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:47.017750 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:47.017965 (+ 215us) server_negotiation.cc:299] Negotiation successful
0926 06:22:47.018033 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 06:22:47.021770 18328 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.202.188" port: 41027 } attempt: 2
Sep 26 06:22:47 dist-test-slave-pd0t krb5kdc[18247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403766, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for kudu/127.4.202.188@KRBTEST.COM
I20260926 06:22:47.028294 18330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:47.021988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:41027 (local address 127.0.0.1:42130)
0926 06:22:47.022027 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:47.022035 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:47.022066 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:47.022380 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:47.022381 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:47.022512 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:22:47.022541 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:47.022543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:47.023182 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:47.023183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:47.023788 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:47.023790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:47.024222 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:47.024223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:47.024275 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:47.024685 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:22:47.024690 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:22:47.025776 (+ 1086us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:22:47.027292 (+ 1516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:47.027293 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:47.027294 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:47.027459 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:47.027667 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:47.027668 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:47.027669 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:47.027724 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:47.027990 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:22:47.027991 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:22:47.028030 (+ 39us) client_negotiation.cc:769] Sending connection context
0926 06:22:47.028124 (+ 94us) client_negotiation.cc:240] Negotiation successful
0926 06:22:47.028209 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260926 06:22:47.028290 18481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:47.022137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42130 (local address 127.4.202.188:41027)
0926 06:22:47.022200 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:47.022202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:47.022209 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:22:47.022226 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:47.022227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:47.022235 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:47.022372 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:47.022665 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:47.023089 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:47.023885 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:47.024139 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:47.024421 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:47.025868 (+ 1447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:47.025870 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:47.025870 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:47.025877 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:47.027191 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:47.027545 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:47.027546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:47.027546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:47.027602 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:47.027804 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:47.027805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:47.027805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:47.027912 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:47.027982 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:47.028149 (+ 167us) server_negotiation.cc:299] Negotiation successful
0926 06:22:47.028209 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 06:22:47.029670 18328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcab76b12dbd45d7b6c052b0b49e106c: Bootstrap starting.
I20260926 06:22:47.030390 18328 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dcab76b12dbd45d7b6c052b0b49e106c: Neither blocks nor log segments found. Creating new log.
I20260926 06:22:47.030768 18328 log.cc:824] T 00000000000000000000000000000000 P dcab76b12dbd45d7b6c052b0b49e106c: Log is configured to *not* fsync() on all Append() calls
I20260926 06:22:47.031859 18328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcab76b12dbd45d7b6c052b0b49e106c: No bootstrap required, opened a new log
I20260926 06:22:47.033844 18328 raft_consensus.cc:348] T 00000000000000000000000000000000 P dcab76b12dbd45d7b6c052b0b49e106c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcab76b12dbd45d7b6c052b0b49e106c" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43075 } } peers { permanent_uuid: "9567db560fc248129de3bfa164c49794" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 39279 } } peers { permanent_uuid: "cf1268b6fba945eba3eb5b1f8f4a292f" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 41027 } }
I20260926 06:22:47.034003 18328 raft_consensus.cc:374] T 00000000000000000000000000000000 P dcab76b12dbd45d7b6c052b0b49e106c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:22:47.034056 18328 raft_consensus.cc:729] T 00000000000000000000000000000000 P dcab76b12dbd45d7b6c052b0b49e106c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcab76b12dbd45d7b6c052b0b49e106c, State: Initialized, Role: FOLLOWER
I20260926 06:22:47.034204 18328 consensus_queue.cc:260] T 00000000000000000000000000000000 P dcab76b12dbd45d7b6c052b0b49e106c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcab76b12dbd45d7b6c052b0b49e106c" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43075 } } peers { permanent_uuid: "9567db560fc248129de3bfa164c49794" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 39279 } } peers { permanent_uuid: "cf1268b6fba945eba3eb5b1f8f4a292f" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 41027 } }
I20260926 06:22:47.034498 18328 sys_catalog.cc:564] T 00000000000000000000000000000000 P dcab76b12dbd45d7b6c052b0b49e106c [sys.catalog]: configured and running, proceeding with master startup.
W20260926 06:22:47.036981 18531 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dcab76b12dbd45d7b6c052b0b49e106c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:22:47.037083 18531 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:22:47.039135 18260 master_runner.cc:263] No leader master found from master dcab76b12dbd45d7b6c052b0b49e106c
I20260926 06:22:47.039183 18260 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 06:22:47.046473 18520 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcab76b12dbd45d7b6c052b0b49e106c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcab76b12dbd45d7b6c052b0b49e106c" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43075 } } peers { permanent_uuid: "9567db560fc248129de3bfa164c49794" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 39279 } } peers { permanent_uuid: "cf1268b6fba945eba3eb5b1f8f4a292f" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 41027 } } }
I20260926 06:22:47.046658 18520 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcab76b12dbd45d7b6c052b0b49e106c [sys.catalog]: This master's current role is: FOLLOWER
W20260926 06:22:47.071758 18356 tablet.cc:2369] T 00000000000000000000000000000000 P 9567db560fc248129de3bfa164c49794: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 06:22:47.178707 18281 tablet.cc:2369] T 00000000000000000000000000000000 P dcab76b12dbd45d7b6c052b0b49e106c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 06:22:47.211973 18431 tablet.cc:2369] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:22:47.268042 18499 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:22:47.268181 18499 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcab76b12dbd45d7b6c052b0b49e106c" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43075 } } peers { permanent_uuid: "9567db560fc248129de3bfa164c49794" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 39279 } } peers { permanent_uuid: "cf1268b6fba945eba3eb5b1f8f4a292f" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 41027 } }
I20260926 06:22:47.268420 18499 leader_election.cc:288] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dcab76b12dbd45d7b6c052b0b49e106c (127.4.202.190:43075), 9567db560fc248129de3bfa164c49794 (127.4.202.189:39279)
I20260926 06:22:47.268752 18377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cf1268b6fba945eba3eb5b1f8f4a292f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9567db560fc248129de3bfa164c49794" is_pre_election: true
I20260926 06:22:47.268784 18302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cf1268b6fba945eba3eb5b1f8f4a292f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcab76b12dbd45d7b6c052b0b49e106c" is_pre_election: true
I20260926 06:22:47.268922 18377 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9567db560fc248129de3bfa164c49794 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf1268b6fba945eba3eb5b1f8f4a292f in term 0.
I20260926 06:22:47.268922 18302 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dcab76b12dbd45d7b6c052b0b49e106c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf1268b6fba945eba3eb5b1f8f4a292f in term 0.
I20260926 06:22:47.269104 18426 leader_election.cc:302] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9567db560fc248129de3bfa164c49794, cf1268b6fba945eba3eb5b1f8f4a292f; no voters:
I20260926 06:22:47.269258 18499 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:22:47.269341 18499 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:22:47.269382 18499 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:22:47.270078 18499 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcab76b12dbd45d7b6c052b0b49e106c" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43075 } } peers { permanent_uuid: "9567db560fc248129de3bfa164c49794" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 39279 } } peers { permanent_uuid: "cf1268b6fba945eba3eb5b1f8f4a292f" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 41027 } }
I20260926 06:22:47.270207 18499 leader_election.cc:288] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f [CANDIDATE]: Term 1 election: Requested vote from peers dcab76b12dbd45d7b6c052b0b49e106c (127.4.202.190:43075), 9567db560fc248129de3bfa164c49794 (127.4.202.189:39279)
I20260926 06:22:47.270457 18302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cf1268b6fba945eba3eb5b1f8f4a292f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcab76b12dbd45d7b6c052b0b49e106c"
I20260926 06:22:47.270457 18377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cf1268b6fba945eba3eb5b1f8f4a292f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9567db560fc248129de3bfa164c49794"
I20260926 06:22:47.270550 18377 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9567db560fc248129de3bfa164c49794 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:22:47.270550 18302 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dcab76b12dbd45d7b6c052b0b49e106c [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:22:47.271447 18302 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dcab76b12dbd45d7b6c052b0b49e106c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf1268b6fba945eba3eb5b1f8f4a292f in term 1.
I20260926 06:22:47.271471 18377 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9567db560fc248129de3bfa164c49794 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf1268b6fba945eba3eb5b1f8f4a292f in term 1.
I20260926 06:22:47.271648 18426 leader_election.cc:302] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9567db560fc248129de3bfa164c49794, cf1268b6fba945eba3eb5b1f8f4a292f; no voters:
I20260926 06:22:47.271723 18499 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:22:47.271905 18499 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f [term 1 LEADER]: Becoming Leader. State: Replica: cf1268b6fba945eba3eb5b1f8f4a292f, State: Running, Role: LEADER
I20260926 06:22:47.272023 18499 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcab76b12dbd45d7b6c052b0b49e106c" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43075 } } peers { permanent_uuid: "9567db560fc248129de3bfa164c49794" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 39279 } } peers { permanent_uuid: "cf1268b6fba945eba3eb5b1f8f4a292f" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 41027 } }
I20260926 06:22:47.272492 18533 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf1268b6fba945eba3eb5b1f8f4a292f. Latest consensus state: current_term: 1 leader_uuid: "cf1268b6fba945eba3eb5b1f8f4a292f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcab76b12dbd45d7b6c052b0b49e106c" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43075 } } peers { permanent_uuid: "9567db560fc248129de3bfa164c49794" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 39279 } } peers { permanent_uuid: "cf1268b6fba945eba3eb5b1f8f4a292f" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 41027 } } }
I20260926 06:22:47.272580 18533 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f [sys.catalog]: This master's current role is: LEADER
I20260926 06:22:47.272773 18535 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:22:47.272953 18535 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:22:47.274477 18377 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9567db560fc248129de3bfa164c49794 [term 1 FOLLOWER]: Refusing update from remote peer cf1268b6fba945eba3eb5b1f8f4a292f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:22:47.274500 18302 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dcab76b12dbd45d7b6c052b0b49e106c [term 1 FOLLOWER]: Refusing update from remote peer cf1268b6fba945eba3eb5b1f8f4a292f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:22:47.274833 18533 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9567db560fc248129de3bfa164c49794" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 39279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:22:47.274955 18499 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dcab76b12dbd45d7b6c052b0b49e106c" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:22:47.276041 18485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9567db560fc248129de3bfa164c49794 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf1268b6fba945eba3eb5b1f8f4a292f. Latest consensus state: current_term: 1 leader_uuid: "cf1268b6fba945eba3eb5b1f8f4a292f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcab76b12dbd45d7b6c052b0b49e106c" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43075 } } peers { permanent_uuid: "9567db560fc248129de3bfa164c49794" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 39279 } } peers { permanent_uuid: "cf1268b6fba945eba3eb5b1f8f4a292f" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 41027 } } }
I20260926 06:22:47.276070 18520 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcab76b12dbd45d7b6c052b0b49e106c [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf1268b6fba945eba3eb5b1f8f4a292f. Latest consensus state: current_term: 1 leader_uuid: "cf1268b6fba945eba3eb5b1f8f4a292f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcab76b12dbd45d7b6c052b0b49e106c" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43075 } } peers { permanent_uuid: "9567db560fc248129de3bfa164c49794" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 39279 } } peers { permanent_uuid: "cf1268b6fba945eba3eb5b1f8f4a292f" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 41027 } } }
I20260926 06:22:47.276127 18485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9567db560fc248129de3bfa164c49794 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:22:47.276139 18520 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcab76b12dbd45d7b6c052b0b49e106c [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:22:47.276896 18499 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cf1268b6fba945eba3eb5b1f8f4a292f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcab76b12dbd45d7b6c052b0b49e106c" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43075 } } peers { permanent_uuid: "9567db560fc248129de3bfa164c49794" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 39279 } } peers { permanent_uuid: "cf1268b6fba945eba3eb5b1f8f4a292f" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 41027 } } }
I20260926 06:22:47.276908 18533 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cf1268b6fba945eba3eb5b1f8f4a292f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcab76b12dbd45d7b6c052b0b49e106c" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43075 } } peers { permanent_uuid: "9567db560fc248129de3bfa164c49794" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 39279 } } peers { permanent_uuid: "cf1268b6fba945eba3eb5b1f8f4a292f" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 41027 } } }
I20260926 06:22:47.276986 18499 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f [sys.catalog]: This master's current role is: LEADER
I20260926 06:22:47.277007 18533 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f [sys.catalog]: This master's current role is: LEADER
I20260926 06:22:47.277120 18520 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcab76b12dbd45d7b6c052b0b49e106c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cf1268b6fba945eba3eb5b1f8f4a292f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcab76b12dbd45d7b6c052b0b49e106c" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43075 } } peers { permanent_uuid: "9567db560fc248129de3bfa164c49794" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 39279 } } peers { permanent_uuid: "cf1268b6fba945eba3eb5b1f8f4a292f" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 41027 } } }
I20260926 06:22:47.277127 18485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9567db560fc248129de3bfa164c49794 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cf1268b6fba945eba3eb5b1f8f4a292f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcab76b12dbd45d7b6c052b0b49e106c" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 43075 } } peers { permanent_uuid: "9567db560fc248129de3bfa164c49794" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 39279 } } peers { permanent_uuid: "cf1268b6fba945eba3eb5b1f8f4a292f" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 41027 } } }
I20260926 06:22:47.277184 18520 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcab76b12dbd45d7b6c052b0b49e106c [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:22:47.277184 18485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9567db560fc248129de3bfa164c49794 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:22:47.277418 18535 catalog_manager.cc:1347] Generated new cluster ID: e4aabed255084383a8989ef95d14a700
I20260926 06:22:47.277470 18535 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:22:47.277449 18537 mvcc.cc:205] Tried to move back new op lower bound from 7333493830753263616 to 7333493830746746880. Current Snapshot: MvccSnapshot[applied={T|T < 7333493830753263616}]
I20260926 06:22:47.277522 18538 mvcc.cc:205] Tried to move back new op lower bound from 7333493830753263616 to 7333493830746746880. Current Snapshot: MvccSnapshot[applied={T|T < 7333493830753263616 or (T in {7333493830753263616})}]
I20260926 06:22:47.295485 18535 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:22:47.296041 18535 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:22:47.301395 18535 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf1268b6fba945eba3eb5b1f8f4a292f: Generated new TSK 0
I20260926 06:22:47.301568 18535 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:22:47.987165 18496 catalog_manager.cc:1259] Loaded cluster ID: e4aabed255084383a8989ef95d14a700
I20260926 06:22:47.987236 18496 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9567db560fc248129de3bfa164c49794: loading cluster ID for follower catalog manager: success
I20260926 06:22:47.987986 18496 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9567db560fc248129de3bfa164c49794: acquiring CA information for follower catalog manager: success
I20260926 06:22:47.988238 18496 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9567db560fc248129de3bfa164c49794: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 06:22:48.037554 18531 catalog_manager.cc:1259] Loaded cluster ID: e4aabed255084383a8989ef95d14a700
I20260926 06:22:48.037623 18531 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dcab76b12dbd45d7b6c052b0b49e106c: loading cluster ID for follower catalog manager: success
I20260926 06:22:48.038826 18531 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dcab76b12dbd45d7b6c052b0b49e106c: acquiring CA information for follower catalog manager: success
I20260926 06:22:48.039151 18531 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dcab76b12dbd45d7b6c052b0b49e106c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 06:22:48.366024 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 18260
I20260926 06:22:48.371124 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 18335
I20260926 06:22:48.376298 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 18410
2026-09-26T06:22:48Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2943 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:22:48 dist-test-slave-pd0t krb5kdc[18551](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:22:48 dist-test-slave-pd0t krb5kdc[18551](info): set up 2 sockets
Sep 26 06:22:48 dist-test-slave-pd0t krb5kdc[18551](info): commencing operation
krb5kdc: starting...
W20260926 06:22:49.414775 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:22:49 dist-test-slave-pd0t krb5kdc[18551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:22:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:22:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.202.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
I20260926 06:22:49.525343 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:45793
--webserver_interface=127.4.202.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:45747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:45793
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:22:49.603030 18564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:22:49.603237 18564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:22:49.603268 18564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:22:49.604318 18564 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:22:49.604378 18564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:22:49.604401 18564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:22:49.604429 18564 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:22:49.604460 18564 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:22:49.605856 18564 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:45747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:45793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:45793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:22:49.606110 18564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:22:49.606315 18564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:22:49.610839 18572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:49.610894 18569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:49.610848 18570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:22:49.611299 18564 server_base.cc:1034] running on GCE node
I20260926 06:22:49.611538 18564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:22:49.611802 18564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:22:49.612968 18564 hybrid_clock.cc:648] HybridClock initialized: now 1790403769612964 us; error 35 us; skew 500 ppm
Sep 26 06:22:49 dist-test-slave-pd0t krb5kdc[18551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403769, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:22:49.615427 18564 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:22:49.615829 18564 webserver.cc:460] Webserver started at http://127.4.202.190:39799/ using document root <none> and password file <none>
I20260926 06:22:49.616044 18564 fs_manager.cc:359] Metadata directory not provided
I20260926 06:22:49.616111 18564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:22:49.616254 18564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:22:49.617246 18564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "81220df2c070496aaa8c9f18bf8909bc"
format_stamp: "Formatted at 2026-09-26 06:22:49 on dist-test-slave-pd0t"
server_key: "9c778f14a69f854a685d90f1e6de6aac"
server_key_iv: "075d8136db82c631a08c618b223d0708"
server_key_version: "encryptionkey@0"
I20260926 06:22:49.617606 18564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "81220df2c070496aaa8c9f18bf8909bc"
format_stamp: "Formatted at 2026-09-26 06:22:49 on dist-test-slave-pd0t"
server_key: "9c778f14a69f854a685d90f1e6de6aac"
server_key_iv: "075d8136db82c631a08c618b223d0708"
server_key_version: "encryptionkey@0"
I20260926 06:22:49.618925 18564 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:22:49.619655 18579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:49.619818 18564 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:22:49.619917 18564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "81220df2c070496aaa8c9f18bf8909bc"
format_stamp: "Formatted at 2026-09-26 06:22:49 on dist-test-slave-pd0t"
server_key: "9c778f14a69f854a685d90f1e6de6aac"
server_key_iv: "075d8136db82c631a08c618b223d0708"
server_key_version: "encryptionkey@0"
I20260926 06:22:49.620002 18564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:22:49.638790 18564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:22:49.640898 18564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:22:49.641052 18564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:22:49.644992 18564 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:45793
I20260926 06:22:49.645021 18631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:45793 every 8 connection(s)
I20260926 06:22:49.645354 18564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:22:49.646123 18632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:22:49.648326 18632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81220df2c070496aaa8c9f18bf8909bc: Bootstrap starting.
I20260926 06:22:49.648813 18632 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81220df2c070496aaa8c9f18bf8909bc: Neither blocks nor log segments found. Creating new log.
I20260926 06:22:49.649027 18632 log.cc:824] T 00000000000000000000000000000000 P 81220df2c070496aaa8c9f18bf8909bc: Log is configured to *not* fsync() on all Append() calls
I20260926 06:22:49.649322 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 18564
I20260926 06:22:49.649394 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:22:49.649609 4906 external_mini_cluster.cc:1428] Setting key b65da53e8cb5af604277badbccf44086
I20260926 06:22:49.649752 18632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81220df2c070496aaa8c9f18bf8909bc: No bootstrap required, opened a new log
I20260926 06:22:49.652077 18632 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81220df2c070496aaa8c9f18bf8909bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81220df2c070496aaa8c9f18bf8909bc" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45793 } }
I20260926 06:22:49.652225 18632 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81220df2c070496aaa8c9f18bf8909bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:22:49.652281 18632 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81220df2c070496aaa8c9f18bf8909bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81220df2c070496aaa8c9f18bf8909bc, State: Initialized, Role: FOLLOWER
I20260926 06:22:49.652491 18632 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81220df2c070496aaa8c9f18bf8909bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81220df2c070496aaa8c9f18bf8909bc" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45793 } }
I20260926 06:22:49.652586 18632 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81220df2c070496aaa8c9f18bf8909bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:22:49.652630 18632 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81220df2c070496aaa8c9f18bf8909bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:22:49.652678 18632 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81220df2c070496aaa8c9f18bf8909bc [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:22:49.653177 18632 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81220df2c070496aaa8c9f18bf8909bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81220df2c070496aaa8c9f18bf8909bc" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45793 } }
I20260926 06:22:49.653302 18632 leader_election.cc:302] T 00000000000000000000000000000000 P 81220df2c070496aaa8c9f18bf8909bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81220df2c070496aaa8c9f18bf8909bc; no voters:
I20260926 06:22:49.653469 18632 leader_election.cc:288] T 00000000000000000000000000000000 P 81220df2c070496aaa8c9f18bf8909bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:22:49.653496 18637 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81220df2c070496aaa8c9f18bf8909bc [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:22:49.653658 18637 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81220df2c070496aaa8c9f18bf8909bc [term 1 LEADER]: Becoming Leader. State: Replica: 81220df2c070496aaa8c9f18bf8909bc, State: Running, Role: LEADER
I20260926 06:22:49.653693 18632 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81220df2c070496aaa8c9f18bf8909bc [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:22:49.653791 18637 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81220df2c070496aaa8c9f18bf8909bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81220df2c070496aaa8c9f18bf8909bc" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45793 } }
I20260926 06:22:49.654140 18637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81220df2c070496aaa8c9f18bf8909bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81220df2c070496aaa8c9f18bf8909bc. Latest consensus state: current_term: 1 leader_uuid: "81220df2c070496aaa8c9f18bf8909bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81220df2c070496aaa8c9f18bf8909bc" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45793 } } }
I20260926 06:22:49.654148 18638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81220df2c070496aaa8c9f18bf8909bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81220df2c070496aaa8c9f18bf8909bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81220df2c070496aaa8c9f18bf8909bc" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45793 } } }
I20260926 06:22:49.654296 18638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81220df2c070496aaa8c9f18bf8909bc [sys.catalog]: This master's current role is: LEADER
I20260926 06:22:49.654505 18637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81220df2c070496aaa8c9f18bf8909bc [sys.catalog]: This master's current role is: LEADER
I20260926 06:22:49.654897 18645 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:22:49.655418 18645 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:22:49.657085 18645 catalog_manager.cc:1347] Generated new cluster ID: 1c0b8bc2626c45fc9bab8cdc2ca0c425
I20260926 06:22:49.657146 18645 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:22:49 dist-test-slave-pd0t krb5kdc[18551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:22:49.661844 18636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:49.650051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34344 (local address 127.4.202.190:45793)
0926 06:22:49.650203 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:49.650211 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:49.650244 (+ 33us) server_negotiation.cc:407] Connection header received
0926 06:22:49.651064 (+ 820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:49.651071 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:49.651116 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:49.651218 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:49.651982 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:49.653316 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:49.654429 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:49.655146 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:49.655238 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:49.658172 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:49.658176 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:49.658176 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:49.658187 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:49.660051 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:49.660447 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:49.660450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:49.660450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:49.660543 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:49.660756 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:49.660756 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:49.660757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:49.661020 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:49.661157 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:49.661528 (+ 371us) server_negotiation.cc:299] Negotiation successful
0926 06:22:49.661651 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":68,"threads_started":1}
I20260926 06:22:49.662803 4906 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:22:49.664395 18645 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:22:49.665071 18645 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:22:49.671581 18645 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81220df2c070496aaa8c9f18bf8909bc: Generated new TSK 0
I20260926 06:22:49.671852 18645 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:22:49.716355 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 18564
2026-09-26T06:22:49Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1337 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:22:49 dist-test-slave-pd0t krb5kdc[18659](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:22:49 dist-test-slave-pd0t krb5kdc[18659](info): set up 2 sockets
Sep 26 06:22:49 dist-test-slave-pd0t krb5kdc[18659](info): commencing operation
krb5kdc: starting...
W20260926 06:22:50.735157 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.002s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:22:50 dist-test-slave-pd0t krb5kdc[18659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:22:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:22:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.202.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
I20260926 06:22:50.848654 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:42879
--webserver_interface=127.4.202.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:34237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:42879
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:22:50.925647 18672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:22:50.925841 18672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:22:50.925864 18672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:22:50.926991 18672 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:22:50.927031 18672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:22:50.927044 18672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:22:50.927057 18672 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:22:50.927100 18672 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:22:50.928566 18672 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:34237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:42879
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:42879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:22:50.928807 18672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:22:50.929018 18672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:22:50.933627 18677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:50.933615 18680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:50.933615 18678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:22:50.934015 18672 server_base.cc:1034] running on GCE node
I20260926 06:22:50.934260 18672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:22:50.934545 18672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:22:50.935720 18672 hybrid_clock.cc:648] HybridClock initialized: now 1790403770935707 us; error 25 us; skew 500 ppm
Sep 26 06:22:50 dist-test-slave-pd0t krb5kdc[18659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403770, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:22:50.938019 18672 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:22:50.938488 18672 webserver.cc:460] Webserver started at http://127.4.202.190:39647/ using document root <none> and password file <none>
I20260926 06:22:50.938738 18672 fs_manager.cc:359] Metadata directory not provided
I20260926 06:22:50.938804 18672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:22:50.938934 18672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:22:50.939885 18672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "43b852defc4e495187ed2c8cb041f43f"
format_stamp: "Formatted at 2026-09-26 06:22:50 on dist-test-slave-pd0t"
server_key: "e6277eec2f5b03889b4c1fa590e30de2"
server_key_iv: "e1463e74589144647f9100985fcf9009"
server_key_version: "encryptionkey@0"
I20260926 06:22:50.940234 18672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "43b852defc4e495187ed2c8cb041f43f"
format_stamp: "Formatted at 2026-09-26 06:22:50 on dist-test-slave-pd0t"
server_key: "e6277eec2f5b03889b4c1fa590e30de2"
server_key_iv: "e1463e74589144647f9100985fcf9009"
server_key_version: "encryptionkey@0"
I20260926 06:22:50.941455 18672 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 06:22:50.942103 18687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:50.942287 18672 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:50.942416 18672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "43b852defc4e495187ed2c8cb041f43f"
format_stamp: "Formatted at 2026-09-26 06:22:50 on dist-test-slave-pd0t"
server_key: "e6277eec2f5b03889b4c1fa590e30de2"
server_key_iv: "e1463e74589144647f9100985fcf9009"
server_key_version: "encryptionkey@0"
I20260926 06:22:50.942502 18672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:22:50.959851 18672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:22:50.961988 18672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:22:50.962139 18672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:22:50.966122 18672 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:42879
I20260926 06:22:50.966174 18739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:42879 every 8 connection(s)
I20260926 06:22:50.966516 18672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:22:50.967473 18740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:22:50.969636 18740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f: Bootstrap starting.
I20260926 06:22:50.970180 18740 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f: Neither blocks nor log segments found. Creating new log.
I20260926 06:22:50.970448 18740 log.cc:824] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f: Log is configured to *not* fsync() on all Append() calls
I20260926 06:22:50.971100 18740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f: No bootstrap required, opened a new log
I20260926 06:22:50.972779 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 18672
I20260926 06:22:50.972711 18740 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b852defc4e495187ed2c8cb041f43f" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42879 } }
I20260926 06:22:50.972868 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:22:50.972882 18740 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:22:50.972935 18740 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43b852defc4e495187ed2c8cb041f43f, State: Initialized, Role: FOLLOWER
I20260926 06:22:50.973074 4906 external_mini_cluster.cc:1428] Setting key cc0d54c6057129a2b166358fbac927c8
I20260926 06:22:50.973070 18740 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b852defc4e495187ed2c8cb041f43f" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42879 } }
I20260926 06:22:50.973151 18740 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:22:50.973197 18740 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:22:50.973255 18740 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:22:50.974305 18740 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b852defc4e495187ed2c8cb041f43f" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42879 } }
I20260926 06:22:50.974392 18740 leader_election.cc:302] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43b852defc4e495187ed2c8cb041f43f; no voters:
I20260926 06:22:50.974555 18740 leader_election.cc:288] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:22:50.974629 18745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:22:50.974819 18745 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [term 1 LEADER]: Becoming Leader. State: Replica: 43b852defc4e495187ed2c8cb041f43f, State: Running, Role: LEADER
I20260926 06:22:50.974884 18740 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:22:50.974963 18745 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b852defc4e495187ed2c8cb041f43f" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42879 } }
I20260926 06:22:50.975297 18745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43b852defc4e495187ed2c8cb041f43f. Latest consensus state: current_term: 1 leader_uuid: "43b852defc4e495187ed2c8cb041f43f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b852defc4e495187ed2c8cb041f43f" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42879 } } }
I20260926 06:22:50.975373 18745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [sys.catalog]: This master's current role is: LEADER
I20260926 06:22:50.975996 18753 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:22:50.976468 18746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43b852defc4e495187ed2c8cb041f43f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b852defc4e495187ed2c8cb041f43f" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42879 } } }
I20260926 06:22:50.976613 18746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [sys.catalog]: This master's current role is: LEADER
I20260926 06:22:50.976624 18753 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:22:50.978190 18753 catalog_manager.cc:1347] Generated new cluster ID: ce89315171d34522a610eed6b3b403d5
I20260926 06:22:50.978245 18753 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:22:50 dist-test-slave-pd0t krb5kdc[18659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:22:50.984073 18744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:50.973500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49192 (local address 127.4.202.190:42879)
0926 06:22:50.973710 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:50.973720 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:50.973762 (+ 42us) server_negotiation.cc:407] Connection header received
0926 06:22:50.973901 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:50.973910 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:50.973976 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:50.974142 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:50.975001 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:50.976528 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:50.977342 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:50.977849 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:50.977952 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:50.980206 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:50.980210 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:50.980210 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:50.980221 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:50.981839 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:50.982214 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:50.982217 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:50.982218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:50.982365 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:50.982597 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:50.982598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:50.982598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:50.983373 (+ 775us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:50.983547 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:50.983657 (+ 110us) server_negotiation.cc:299] Negotiation successful
0926 06:22:50.983908 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":80,"threads_started":1}
I20260926 06:22:50.984464 18753 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:22:50.985152 18753 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:22:50.991323 18753 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f: Generated new TSK 0
I20260926 06:22:50.991529 18753 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.202.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
I20260926 06:22:51.023034 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.129:0
--local_ip_for_outbound_sockets=127.4.202.129
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:42879
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:34237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:22:51.102928 18766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:22:51.103116 18766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:22:51.103139 18766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:22:51.104216 18766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:22:51.104252 18766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:22:51.104276 18766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.129
I20260926 06:22:51.105609 18766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:34237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:42879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.129
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:22:51.105777 18766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:22:51.105942 18766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:22:51.110704 18771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:51.110704 18774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:51.110715 18772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:22:51.110989 18766 server_base.cc:1034] running on GCE node
I20260926 06:22:51.111163 18766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:22:51.111348 18766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:22:51.112509 18766 hybrid_clock.cc:648] HybridClock initialized: now 1790403771112498 us; error 29 us; skew 500 ppm
Sep 26 06:22:51 dist-test-slave-pd0t krb5kdc[18659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403771, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:22:51.114915 18766 init.cc:377] Logged in from keytab as kudu/127.4.202.129@KRBTEST.COM (short username kudu)
I20260926 06:22:51.115360 18766 webserver.cc:460] Webserver started at http://127.4.202.129:46665/ using document root <none> and password file <none>
I20260926 06:22:51.115600 18766 fs_manager.cc:359] Metadata directory not provided
I20260926 06:22:51.115666 18766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:22:51.115816 18766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:22:51.116763 18766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/data/instance:
uuid: "42e3a388aa6a4797afbbc09559b9658f"
format_stamp: "Formatted at 2026-09-26 06:22:51 on dist-test-slave-pd0t"
server_key: "9f573e3bc6b4a7b6d5691861c6996f7f"
server_key_iv: "90b3ac96a9e0d75ff77cf2d53635340a"
server_key_version: "encryptionkey@0"
I20260926 06:22:51.117127 18766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance:
uuid: "42e3a388aa6a4797afbbc09559b9658f"
format_stamp: "Formatted at 2026-09-26 06:22:51 on dist-test-slave-pd0t"
server_key: "9f573e3bc6b4a7b6d5691861c6996f7f"
server_key_iv: "90b3ac96a9e0d75ff77cf2d53635340a"
server_key_version: "encryptionkey@0"
I20260926 06:22:51.118378 18766 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 06:22:51.119124 18781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:51.119275 18766 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:51.119367 18766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/wal
uuid: "42e3a388aa6a4797afbbc09559b9658f"
format_stamp: "Formatted at 2026-09-26 06:22:51 on dist-test-slave-pd0t"
server_key: "9f573e3bc6b4a7b6d5691861c6996f7f"
server_key_iv: "90b3ac96a9e0d75ff77cf2d53635340a"
server_key_version: "encryptionkey@0"
I20260926 06:22:51.119448 18766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:22:51.133713 18766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:22:51.135460 18766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:22:51.135614 18766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:22:51.135840 18766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:22:51.136132 18766 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:22:51.136184 18766 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:51.136216 18766 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:22:51.136243 18766 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:51.141713 18766 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.129:46793
I20260926 06:22:51.141810 18894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.129:46793 every 8 connection(s)
I20260926 06:22:51.142068 18766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
I20260926 06:22:51.147173 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 18766
I20260926 06:22:51.147318 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance
I20260926 06:22:51.147497 4906 external_mini_cluster.cc:1428] Setting key b57d1411ec9e8d9cff43324becb34555
Sep 26 06:22:51 dist-test-slave-pd0t krb5kdc[18659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403771, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:22:51.152366 18897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:51.142756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.129:45281)
0926 06:22:51.142888 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:51.142934 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:51.143411 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:51.143827 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:51.143830 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:51.144450 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:22:51.144694 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:51.144699 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.145476 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.145477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:51.146303 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:51.146305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.146781 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.146782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:51.146906 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:51.147439 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:22:51.147446 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:22:51.149324 (+ 1878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:22:51.151101 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:51.151107 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:51.151109 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:51.151321 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:51.151519 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:51.151520 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:51.151520 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:51.151611 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:51.151892 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:22:51.151898 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:22:51.152023 (+ 125us) client_negotiation.cc:769] Sending connection context
0926 06:22:51.152102 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 06:22:51.152210 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":62,"threads_started":1}
I20260926 06:22:51.152390 18744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:51.142791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:45281 (local address 127.4.202.190:42879)
0926 06:22:51.142837 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:51.142840 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:51.143015 (+ 175us) server_negotiation.cc:407] Connection header received
0926 06:22:51.143673 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:51.143674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:51.143689 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:51.143742 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:51.144798 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.145380 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.146403 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.146695 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.146769 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:51.149415 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:51.149416 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:51.149416 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:51.149422 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:51.151015 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:51.151393 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:51.151394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:51.151395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:51.151453 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:51.151676 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:51.151677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:51.151677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:51.151819 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:51.151892 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:51.152152 (+ 260us) server_negotiation.cc:299] Negotiation successful
0926 06:22:51.152284 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:22:51.153139 18895 heartbeater.cc:347] Connected to a master server at 127.4.202.190:42879
I20260926 06:22:51.153239 18895 heartbeater.cc:464] Registering TS with master...
I20260926 06:22:51.153431 18895 heartbeater.cc:511] Master 127.4.202.190:42879 requested a full tablet report, sending...
I20260926 06:22:51.153905 18704 ts_manager.cc:194] Registered new tserver with Master: 42e3a388aa6a4797afbbc09559b9658f (127.4.202.129:46793)
I20260926 06:22:51.154487 18704 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.129@KRBTEST.COM'} at 127.4.202.129:45281
WARNING: no policy specified for kudu/127.4.202.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
I20260926 06:22:51.173271 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.130:0
--local_ip_for_outbound_sockets=127.4.202.130
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:42879
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:34237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:22:51.249406 18900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:22:51.249604 18900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:22:51.249639 18900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:22:51.250731 18900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:22:51.250792 18900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:22:51.250831 18900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.130
I20260926 06:22:51.252170 18900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:34237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:42879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.130
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:22:51.252431 18900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:22:51.252688 18900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 06:22:51.257640 18900 server_base.cc:1034] running on GCE node
W20260926 06:22:51.257776 18908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:51.257794 18906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:51.257824 18905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:22:51.258173 18900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:22:51.258435 18900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:22:51.259589 18900 hybrid_clock.cc:648] HybridClock initialized: now 1790403771259573 us; error 35 us; skew 500 ppm
Sep 26 06:22:51 dist-test-slave-pd0t krb5kdc[18659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403771, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:22:51.261935 18900 init.cc:377] Logged in from keytab as kudu/127.4.202.130@KRBTEST.COM (short username kudu)
I20260926 06:22:51.262400 18900 webserver.cc:460] Webserver started at http://127.4.202.130:41525/ using document root <none> and password file <none>
I20260926 06:22:51.262665 18900 fs_manager.cc:359] Metadata directory not provided
I20260926 06:22:51.262720 18900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:22:51.262871 18900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:22:51.263844 18900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/data/instance:
uuid: "edf6d196a91e4256a94d85a8f82b4cc8"
format_stamp: "Formatted at 2026-09-26 06:22:51 on dist-test-slave-pd0t"
server_key: "41d2b5ca8edfb75534d0651171d8be90"
server_key_iv: "d722cea4dc482c485608577559f94195"
server_key_version: "encryptionkey@0"
I20260926 06:22:51.264179 18900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance:
uuid: "edf6d196a91e4256a94d85a8f82b4cc8"
format_stamp: "Formatted at 2026-09-26 06:22:51 on dist-test-slave-pd0t"
server_key: "41d2b5ca8edfb75534d0651171d8be90"
server_key_iv: "d722cea4dc482c485608577559f94195"
server_key_version: "encryptionkey@0"
I20260926 06:22:51.265419 18900 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:22:51.266185 18915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:51.266362 18900 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:22:51.266438 18900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/wal
uuid: "edf6d196a91e4256a94d85a8f82b4cc8"
format_stamp: "Formatted at 2026-09-26 06:22:51 on dist-test-slave-pd0t"
server_key: "41d2b5ca8edfb75534d0651171d8be90"
server_key_iv: "d722cea4dc482c485608577559f94195"
server_key_version: "encryptionkey@0"
I20260926 06:22:51.266526 18900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:22:51.292153 18900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:22:51.293987 18900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:22:51.294140 18900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:22:51.294361 18900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:22:51.294713 18900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:22:51.294764 18900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:51.294796 18900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:22:51.294826 18900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:51.300582 18900 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.130:43573
I20260926 06:22:51.300624 19028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.130:43573 every 8 connection(s)
I20260926 06:22:51.300909 18900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
Sep 26 06:22:51 dist-test-slave-pd0t krb5kdc[18659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403771, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:22:51.307958 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 18900
I20260926 06:22:51.308037 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance
I20260926 06:22:51.308219 4906 external_mini_cluster.cc:1428] Setting key 6bf89fe0a4f59d7f1efa4f3b5bf294ba
I20260926 06:22:51.309921 18744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:51.301598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:42981 (local address 127.4.202.190:42879)
0926 06:22:51.301693 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:51.301694 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:51.301853 (+ 159us) server_negotiation.cc:407] Connection header received
0926 06:22:51.302302 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:51.302303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:51.302314 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:51.302369 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:51.303280 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.303822 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.304845 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.305043 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.305098 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:51.307044 (+ 1946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:51.307045 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:51.307046 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:51.307051 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:51.308512 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:51.308991 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:51.308993 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:51.308993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:51.309058 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:51.309344 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:51.309345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:51.309345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:51.309442 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:51.309491 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:51.309801 (+ 310us) server_negotiation.cc:299] Negotiation successful
0926 06:22:51.309864 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 06:22:51.310204 19031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:51.301565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.130:42981)
0926 06:22:51.301708 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:51.301759 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:51.302191 (+ 432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:51.302417 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:51.302421 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:51.302992 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:22:51.303163 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:51.303167 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.303944 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.303945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:51.304758 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:51.304761 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.305117 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.305118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:51.305208 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:51.305629 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:22:51.305637 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:22:51.306956 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:22:51.308598 (+ 1642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:51.308604 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:51.308608 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:51.308895 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:51.309137 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:51.309138 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:51.309139 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:51.309279 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:51.309506 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:22:51.309512 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:22:51.309672 (+ 160us) client_negotiation.cc:769] Sending connection context
0926 06:22:51.309778 (+ 106us) client_negotiation.cc:240] Negotiation successful
0926 06:22:51.309948 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":77,"threads_started":1}
I20260926 06:22:51.310751 19029 heartbeater.cc:347] Connected to a master server at 127.4.202.190:42879
I20260926 06:22:51.310851 19029 heartbeater.cc:464] Registering TS with master...
I20260926 06:22:51.311031 19029 heartbeater.cc:511] Master 127.4.202.190:42879 requested a full tablet report, sending...
I20260926 06:22:51.311357 18704 ts_manager.cc:194] Registered new tserver with Master: edf6d196a91e4256a94d85a8f82b4cc8 (127.4.202.130:43573)
I20260926 06:22:51.311725 18704 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.130@KRBTEST.COM'} at 127.4.202.130:42981
WARNING: no policy specified for kudu/127.4.202.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
I20260926 06:22:51.333127 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.131:0
--local_ip_for_outbound_sockets=127.4.202.131
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:42879
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:34237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:22:51.410148 19034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:22:51.410393 19034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:22:51.410427 19034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:22:51.411552 19034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:22:51.411621 19034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:22:51.411664 19034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.131
I20260926 06:22:51.413084 19034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:34237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:42879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.131
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:22:51.413326 19034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:22:51.413537 19034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 06:22:51.418345 19034 server_base.cc:1034] running on GCE node
W20260926 06:22:51.418295 19040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:51.418331 19042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:51.418304 19039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:22:51.419023 19034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:22:51.419265 19034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:22:51.420459 19034 hybrid_clock.cc:648] HybridClock initialized: now 1790403771420442 us; error 28 us; skew 500 ppm
Sep 26 06:22:51 dist-test-slave-pd0t krb5kdc[18659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403771, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:22:51.423102 19034 init.cc:377] Logged in from keytab as kudu/127.4.202.131@KRBTEST.COM (short username kudu)
I20260926 06:22:51.423511 19034 webserver.cc:460] Webserver started at http://127.4.202.131:44251/ using document root <none> and password file <none>
I20260926 06:22:51.423727 19034 fs_manager.cc:359] Metadata directory not provided
I20260926 06:22:51.423801 19034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:22:51.423947 19034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:22:51.424942 19034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/data/instance:
uuid: "5e2dd6c2c8e245038da7f222b8dd7850"
format_stamp: "Formatted at 2026-09-26 06:22:51 on dist-test-slave-pd0t"
server_key: "6e6ba8d54dc704ef4de3bb13e5750299"
server_key_iv: "df546a04578b5850d57da5890577398e"
server_key_version: "encryptionkey@0"
I20260926 06:22:51.425294 19034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance:
uuid: "5e2dd6c2c8e245038da7f222b8dd7850"
format_stamp: "Formatted at 2026-09-26 06:22:51 on dist-test-slave-pd0t"
server_key: "6e6ba8d54dc704ef4de3bb13e5750299"
server_key_iv: "df546a04578b5850d57da5890577398e"
server_key_version: "encryptionkey@0"
I20260926 06:22:51.426545 19034 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:22:51.427347 19049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:51.427517 19034 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:22:51.427610 19034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/wal
uuid: "5e2dd6c2c8e245038da7f222b8dd7850"
format_stamp: "Formatted at 2026-09-26 06:22:51 on dist-test-slave-pd0t"
server_key: "6e6ba8d54dc704ef4de3bb13e5750299"
server_key_iv: "df546a04578b5850d57da5890577398e"
server_key_version: "encryptionkey@0"
I20260926 06:22:51.427695 19034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:22:51.437194 19034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:22:51.438898 19034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:22:51.439043 19034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:22:51.439260 19034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:22:51.439534 19034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:22:51.439585 19034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:51.439623 19034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:22:51.439651 19034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:51.445502 19034 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.131:40851
I20260926 06:22:51.445545 19162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.131:40851 every 8 connection(s)
I20260926 06:22:51.445886 19034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
I20260926 06:22:51.447129 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 19034
I20260926 06:22:51.447227 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance
I20260926 06:22:51.447455 4906 external_mini_cluster.cc:1428] Setting key 444182ff67ed2ec567c99139cf5f28b3
Sep 26 06:22:51 dist-test-slave-pd0t krb5kdc[18659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403771, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:22:51.455021 18744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:51.446684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:44711 (local address 127.4.202.190:42879)
0926 06:22:51.446748 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:51.446751 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:51.446899 (+ 148us) server_negotiation.cc:407] Connection header received
0926 06:22:51.447577 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:51.447578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:51.447590 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:51.447668 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:51.448799 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.449279 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.450246 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.450446 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.450496 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:51.452597 (+ 2101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:51.452599 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:51.452599 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:51.452605 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:51.453781 (+ 1176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:51.454146 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:51.454147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:51.454147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:51.454199 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:51.454437 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:51.454438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:51.454438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:51.454544 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:51.454681 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:51.454899 (+ 218us) server_negotiation.cc:299] Negotiation successful
0926 06:22:51.454961 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 06:22:51.455155 19165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:51.446543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.131:44711)
0926 06:22:51.446784 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:51.446817 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:51.447464 (+ 647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:51.447757 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:51.447769 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:51.448534 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:22:51.448709 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:51.448714 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.449390 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.449391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:51.450171 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:51.450173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.450519 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.450520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:51.450680 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:51.451110 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:22:51.451117 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:22:51.452510 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:22:51.453852 (+ 1342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:51.453858 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:51.453860 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:51.454079 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:51.454263 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:51.454264 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:51.454264 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:51.454382 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:51.454674 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:22:51.454679 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:22:51.454781 (+ 102us) client_negotiation.cc:769] Sending connection context
0926 06:22:51.454864 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 06:22:51.454974 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":107,"threads_started":1}
I20260926 06:22:51.455695 19163 heartbeater.cc:347] Connected to a master server at 127.4.202.190:42879
I20260926 06:22:51.455794 19163 heartbeater.cc:464] Registering TS with master...
I20260926 06:22:51.455986 19163 heartbeater.cc:511] Master 127.4.202.190:42879 requested a full tablet report, sending...
I20260926 06:22:51.456403 18704 ts_manager.cc:194] Registered new tserver with Master: 5e2dd6c2c8e245038da7f222b8dd7850 (127.4.202.131:40851)
I20260926 06:22:51.456766 18704 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.131@KRBTEST.COM'} at 127.4.202.131:44711
I20260926 06:22:51.459163 4906 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:22:51.466090 18744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:51.460526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49202 (local address 127.4.202.190:42879)
0926 06:22:51.460566 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:51.460568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:51.460679 (+ 111us) server_negotiation.cc:407] Connection header received
0926 06:22:51.460730 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:51.460731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:51.460738 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:51.460781 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:51.461268 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.461728 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.462412 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.462643 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.462694 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:51.464055 (+ 1361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:51.464056 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:51.464056 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:51.464062 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:51.465190 (+ 1128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:51.465503 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:51.465503 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:51.465504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:51.465548 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:51.465679 (+ 131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:51.465680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:51.465680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:51.465797 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:51.465867 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:51.465959 (+ 92us) server_negotiation.cc:299] Negotiation successful
0926 06:22:51.466015 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 06:22:51.467787 18704 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49202:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:22:51.469424 18704 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:22:51.475551 19172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:51.471422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:40851 (local address 127.0.0.1:44646)
0926 06:22:51.471539 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:51.471594 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:51.471659 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:51.472135 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:51.472136 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:51.472138 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:22:51.472200 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:51.472202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.473106 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.473107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:51.474311 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:51.474313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.475206 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.475207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:51.475287 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:51.475299 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 06:22:51.475430 (+ 131us) client_negotiation.cc:240] Negotiation successful
0926 06:22:51.475483 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":51,"threads_started":1}
I20260926 06:22:51.475665 19173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:51.471516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44646 (local address 127.4.202.131:40851)
0926 06:22:51.471934 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:51.471938 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:51.471945 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:22:51.471971 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:51.471972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:51.471983 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:51.472082 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:22:51.472336 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.473004 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.474405 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.475121 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.475202 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:51.475258 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:51.475444 (+ 186us) server_negotiation.cc:299] Negotiation successful
0926 06:22:51.475482 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":91,"threads_started":1}
I20260926 06:22:51.475834 19171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:51.471422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:43573 (local address 127.0.0.1:51636)
0926 06:22:51.471706 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:51.471716 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:51.471783 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:51.472049 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:51.472052 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:51.472055 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:22:51.472196 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:51.472199 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.473536 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.473537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:51.474504 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:51.474507 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.475591 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.475592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:51.475657 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:51.475662 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 06:22:51.475715 (+ 53us) client_negotiation.cc:240] Negotiation successful
0926 06:22:51.475757 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":2,"thread_start_us":64,"threads_started":1}
I20260926 06:22:51.476162 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:51.471691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51636 (local address 127.4.202.130:43573)
0926 06:22:51.471875 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:51.471880 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:51.471890 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:22:51.471929 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:51.471932 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:51.471948 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:51.472022 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:22:51.472343 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.473430 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.474618 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.475436 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.475506 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:51.475555 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:51.476029 (+ 474us) server_negotiation.cc:299] Negotiation successful
0926 06:22:51.476075 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":78,"threads_started":1}
I20260926 06:22:51.476468 19176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:51.471738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34218 (local address 127.4.202.129:46793)
0926 06:22:51.472195 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:51.472199 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:51.472209 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:22:51.472294 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:51.472297 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:51.472325 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:51.472396 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:22:51.472590 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.473579 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.474909 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.475676 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.475747 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:51.475799 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:51.476146 (+ 347us) server_negotiation.cc:299] Negotiation successful
0926 06:22:51.476361 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":85,"threads_started":1}
I20260926 06:22:51.476078 19174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:51.471615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:46793 (local address 127.0.0.1:34218)
0926 06:22:51.471852 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:51.471862 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:51.471911 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:51.472472 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:51.472473 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:51.472475 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:22:51.472507 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:51.472508 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.473690 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.473691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:51.474817 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:51.474820 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.475831 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.475832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:51.475930 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:51.475934 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:22:51.475979 (+ 45us) client_negotiation.cc:240] Negotiation successful
0926 06:22:51.476019 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":45,"threads_started":1}
I20260926 06:22:51.476588 19097 tablet_service.cc:1467] Processing CreateTablet for tablet 2e880990ab2a4dbcb874a48301b96ae4 (DEFAULT_TABLE table=test-table [id=b725533c3bbf4f2dac5eb3c1ae7c07f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:22:51.476789 18963 tablet_service.cc:1467] Processing CreateTablet for tablet 2e880990ab2a4dbcb874a48301b96ae4 (DEFAULT_TABLE table=test-table [id=b725533c3bbf4f2dac5eb3c1ae7c07f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:22:51.476953 19097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e880990ab2a4dbcb874a48301b96ae4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:22:51.477066 18963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e880990ab2a4dbcb874a48301b96ae4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:22:51.477164 18829 tablet_service.cc:1467] Processing CreateTablet for tablet 2e880990ab2a4dbcb874a48301b96ae4 (DEFAULT_TABLE table=test-table [id=b725533c3bbf4f2dac5eb3c1ae7c07f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:22:51.477399 18829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e880990ab2a4dbcb874a48301b96ae4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:22:51.480131 19179 tablet_bootstrap.cc:492] T 2e880990ab2a4dbcb874a48301b96ae4 P edf6d196a91e4256a94d85a8f82b4cc8: Bootstrap starting.
I20260926 06:22:51.480221 19177 tablet_bootstrap.cc:492] T 2e880990ab2a4dbcb874a48301b96ae4 P 42e3a388aa6a4797afbbc09559b9658f: Bootstrap starting.
I20260926 06:22:51.480333 19178 tablet_bootstrap.cc:492] T 2e880990ab2a4dbcb874a48301b96ae4 P 5e2dd6c2c8e245038da7f222b8dd7850: Bootstrap starting.
I20260926 06:22:51.480914 19177 tablet_bootstrap.cc:654] T 2e880990ab2a4dbcb874a48301b96ae4 P 42e3a388aa6a4797afbbc09559b9658f: Neither blocks nor log segments found. Creating new log.
I20260926 06:22:51.480914 19179 tablet_bootstrap.cc:654] T 2e880990ab2a4dbcb874a48301b96ae4 P edf6d196a91e4256a94d85a8f82b4cc8: Neither blocks nor log segments found. Creating new log.
I20260926 06:22:51.480943 19178 tablet_bootstrap.cc:654] T 2e880990ab2a4dbcb874a48301b96ae4 P 5e2dd6c2c8e245038da7f222b8dd7850: Neither blocks nor log segments found. Creating new log.
I20260926 06:22:51.481438 19178 log.cc:824] T 2e880990ab2a4dbcb874a48301b96ae4 P 5e2dd6c2c8e245038da7f222b8dd7850: Log is configured to *not* fsync() on all Append() calls
I20260926 06:22:51.481580 19177 log.cc:824] T 2e880990ab2a4dbcb874a48301b96ae4 P 42e3a388aa6a4797afbbc09559b9658f: Log is configured to *not* fsync() on all Append() calls
I20260926 06:22:51.481580 19179 log.cc:824] T 2e880990ab2a4dbcb874a48301b96ae4 P edf6d196a91e4256a94d85a8f82b4cc8: Log is configured to *not* fsync() on all Append() calls
I20260926 06:22:51.482326 19177 tablet_bootstrap.cc:492] T 2e880990ab2a4dbcb874a48301b96ae4 P 42e3a388aa6a4797afbbc09559b9658f: No bootstrap required, opened a new log
I20260926 06:22:51.482347 19178 tablet_bootstrap.cc:492] T 2e880990ab2a4dbcb874a48301b96ae4 P 5e2dd6c2c8e245038da7f222b8dd7850: No bootstrap required, opened a new log
I20260926 06:22:51.482402 19178 ts_tablet_manager.cc:1397] T 2e880990ab2a4dbcb874a48301b96ae4 P 5e2dd6c2c8e245038da7f222b8dd7850: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:22:51.482402 19177 ts_tablet_manager.cc:1397] T 2e880990ab2a4dbcb874a48301b96ae4 P 42e3a388aa6a4797afbbc09559b9658f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:22:51.482483 19179 tablet_bootstrap.cc:492] T 2e880990ab2a4dbcb874a48301b96ae4 P edf6d196a91e4256a94d85a8f82b4cc8: No bootstrap required, opened a new log
I20260926 06:22:51.482549 19179 ts_tablet_manager.cc:1397] T 2e880990ab2a4dbcb874a48301b96ae4 P edf6d196a91e4256a94d85a8f82b4cc8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:22:51.483908 19179 raft_consensus.cc:348] T 2e880990ab2a4dbcb874a48301b96ae4 P edf6d196a91e4256a94d85a8f82b4cc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40851 } } peers { permanent_uuid: "42e3a388aa6a4797afbbc09559b9658f" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 46793 } } peers { permanent_uuid: "edf6d196a91e4256a94d85a8f82b4cc8" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43573 } }
I20260926 06:22:51.484087 19179 raft_consensus.cc:374] T 2e880990ab2a4dbcb874a48301b96ae4 P edf6d196a91e4256a94d85a8f82b4cc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:22:51.484131 19179 raft_consensus.cc:729] T 2e880990ab2a4dbcb874a48301b96ae4 P edf6d196a91e4256a94d85a8f82b4cc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edf6d196a91e4256a94d85a8f82b4cc8, State: Initialized, Role: FOLLOWER
I20260926 06:22:51.484239 19179 consensus_queue.cc:260] T 2e880990ab2a4dbcb874a48301b96ae4 P edf6d196a91e4256a94d85a8f82b4cc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40851 } } peers { permanent_uuid: "42e3a388aa6a4797afbbc09559b9658f" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 46793 } } peers { permanent_uuid: "edf6d196a91e4256a94d85a8f82b4cc8" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43573 } }
I20260926 06:22:51.484356 19178 raft_consensus.cc:348] T 2e880990ab2a4dbcb874a48301b96ae4 P 5e2dd6c2c8e245038da7f222b8dd7850 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40851 } } peers { permanent_uuid: "42e3a388aa6a4797afbbc09559b9658f" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 46793 } } peers { permanent_uuid: "edf6d196a91e4256a94d85a8f82b4cc8" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43573 } }
I20260926 06:22:51.484354 19177 raft_consensus.cc:348] T 2e880990ab2a4dbcb874a48301b96ae4 P 42e3a388aa6a4797afbbc09559b9658f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40851 } } peers { permanent_uuid: "42e3a388aa6a4797afbbc09559b9658f" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 46793 } } peers { permanent_uuid: "edf6d196a91e4256a94d85a8f82b4cc8" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43573 } }
I20260926 06:22:51.484526 19178 raft_consensus.cc:374] T 2e880990ab2a4dbcb874a48301b96ae4 P 5e2dd6c2c8e245038da7f222b8dd7850 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:22:51.484539 19179 ts_tablet_manager.cc:1428] T 2e880990ab2a4dbcb874a48301b96ae4 P edf6d196a91e4256a94d85a8f82b4cc8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:22:51.484565 19029 heartbeater.cc:503] Master 127.4.202.190:42879 was elected leader, sending a full tablet report...
I20260926 06:22:51.484570 19178 raft_consensus.cc:729] T 2e880990ab2a4dbcb874a48301b96ae4 P 5e2dd6c2c8e245038da7f222b8dd7850 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2dd6c2c8e245038da7f222b8dd7850, State: Initialized, Role: FOLLOWER
I20260926 06:22:51.484694 19178 consensus_queue.cc:260] T 2e880990ab2a4dbcb874a48301b96ae4 P 5e2dd6c2c8e245038da7f222b8dd7850 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40851 } } peers { permanent_uuid: "42e3a388aa6a4797afbbc09559b9658f" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 46793 } } peers { permanent_uuid: "edf6d196a91e4256a94d85a8f82b4cc8" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43573 } }
I20260926 06:22:51.484840 19177 raft_consensus.cc:374] T 2e880990ab2a4dbcb874a48301b96ae4 P 42e3a388aa6a4797afbbc09559b9658f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:22:51.484910 19177 raft_consensus.cc:729] T 2e880990ab2a4dbcb874a48301b96ae4 P 42e3a388aa6a4797afbbc09559b9658f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42e3a388aa6a4797afbbc09559b9658f, State: Initialized, Role: FOLLOWER
I20260926 06:22:51.485034 19163 heartbeater.cc:503] Master 127.4.202.190:42879 was elected leader, sending a full tablet report...
I20260926 06:22:51.485018 19177 consensus_queue.cc:260] T 2e880990ab2a4dbcb874a48301b96ae4 P 42e3a388aa6a4797afbbc09559b9658f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40851 } } peers { permanent_uuid: "42e3a388aa6a4797afbbc09559b9658f" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 46793 } } peers { permanent_uuid: "edf6d196a91e4256a94d85a8f82b4cc8" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43573 } }
I20260926 06:22:51.485285 19177 ts_tablet_manager.cc:1428] T 2e880990ab2a4dbcb874a48301b96ae4 P 42e3a388aa6a4797afbbc09559b9658f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 06:22:51.485555 18895 heartbeater.cc:503] Master 127.4.202.190:42879 was elected leader, sending a full tablet report...
I20260926 06:22:51.485918 19178 ts_tablet_manager.cc:1428] T 2e880990ab2a4dbcb874a48301b96ae4 P 5e2dd6c2c8e245038da7f222b8dd7850: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260926 06:22:51.551420 19030 tablet.cc:2369] T 2e880990ab2a4dbcb874a48301b96ae4 P edf6d196a91e4256a94d85a8f82b4cc8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:22:51.632318 19184 raft_consensus.cc:482] T 2e880990ab2a4dbcb874a48301b96ae4 P 5e2dd6c2c8e245038da7f222b8dd7850 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:22:51.632438 19184 raft_consensus.cc:504] T 2e880990ab2a4dbcb874a48301b96ae4 P 5e2dd6c2c8e245038da7f222b8dd7850 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40851 } } peers { permanent_uuid: "42e3a388aa6a4797afbbc09559b9658f" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 46793 } } peers { permanent_uuid: "edf6d196a91e4256a94d85a8f82b4cc8" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43573 } }
I20260926 06:22:51.632822 19184 leader_election.cc:288] T 2e880990ab2a4dbcb874a48301b96ae4 P 5e2dd6c2c8e245038da7f222b8dd7850 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 42e3a388aa6a4797afbbc09559b9658f (127.4.202.129:46793), edf6d196a91e4256a94d85a8f82b4cc8 (127.4.202.130:43573)
I20260926 06:22:51.635846 19165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:51.633022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:46793 (local address 127.4.202.131:49933)
0926 06:22:51.633089 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:51.633103 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:51.633169 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:51.633311 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:51.633312 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:51.633314 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:22:51.633397 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:51.633399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.634080 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.634081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:51.634971 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:51.634974 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.635562 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.635563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:51.635644 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:51.635648 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:22:51.635716 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 06:22:51.635775 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":24}
I20260926 06:22:51.635866 19176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:51.633071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:49933 (local address 127.4.202.129:46793)
0926 06:22:51.633116 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:51.633118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:51.633174 (+ 56us) server_negotiation.cc:407] Connection header received
0926 06:22:51.633236 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:51.633237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:51.633246 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:51.633287 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:22:51.633478 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.634006 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.635057 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.635477 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.635532 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:51.635538 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:51.635744 (+ 206us) server_negotiation.cc:299] Negotiation successful
0926 06:22:51.635781 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 06:22:51.636060 19186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:51.633023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:43573 (local address 127.4.202.131:52771)
0926 06:22:51.633176 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:51.633186 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:51.633229 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:51.633481 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:51.633482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:51.633484 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:22:51.633556 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:51.633557 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.634265 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.634265 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:51.635100 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:51.635102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.635773 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.635774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:51.635839 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:51.635843 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:22:51.635925 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 06:22:51.635968 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":14,"thread_start_us":89,"threads_started":1}
I20260926 06:22:51.636212 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:51.633121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:52771 (local address 127.4.202.130:43573)
0926 06:22:51.633346 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:51.633348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:51.633356 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:22:51.633383 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:51.633384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:51.633392 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:51.633454 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:22:51.633663 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.634183 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.635218 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.635693 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.635759 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:51.635766 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:51.636100 (+ 334us) server_negotiation.cc:299] Negotiation successful
0926 06:22:51.636148 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201}
I20260926 06:22:51.636250 18849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e880990ab2a4dbcb874a48301b96ae4" candidate_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42e3a388aa6a4797afbbc09559b9658f" is_pre_election: true
I20260926 06:22:51.636416 18849 raft_consensus.cc:2454] T 2e880990ab2a4dbcb874a48301b96ae4 P 42e3a388aa6a4797afbbc09559b9658f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2dd6c2c8e245038da7f222b8dd7850 in term 0.
I20260926 06:22:51.636551 18983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e880990ab2a4dbcb874a48301b96ae4" candidate_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edf6d196a91e4256a94d85a8f82b4cc8" is_pre_election: true
I20260926 06:22:51.636710 18983 raft_consensus.cc:2454] T 2e880990ab2a4dbcb874a48301b96ae4 P edf6d196a91e4256a94d85a8f82b4cc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2dd6c2c8e245038da7f222b8dd7850 in term 0.
I20260926 06:22:51.636706 19052 leader_election.cc:302] T 2e880990ab2a4dbcb874a48301b96ae4 P 5e2dd6c2c8e245038da7f222b8dd7850 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42e3a388aa6a4797afbbc09559b9658f, 5e2dd6c2c8e245038da7f222b8dd7850; no voters:
I20260926 06:22:51.636857 19184 raft_consensus.cc:2781] T 2e880990ab2a4dbcb874a48301b96ae4 P 5e2dd6c2c8e245038da7f222b8dd7850 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:22:51.636931 19184 raft_consensus.cc:482] T 2e880990ab2a4dbcb874a48301b96ae4 P 5e2dd6c2c8e245038da7f222b8dd7850 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:22:51.636957 19184 raft_consensus.cc:3037] T 2e880990ab2a4dbcb874a48301b96ae4 P 5e2dd6c2c8e245038da7f222b8dd7850 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:22:51.637640 19184 raft_consensus.cc:504] T 2e880990ab2a4dbcb874a48301b96ae4 P 5e2dd6c2c8e245038da7f222b8dd7850 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40851 } } peers { permanent_uuid: "42e3a388aa6a4797afbbc09559b9658f" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 46793 } } peers { permanent_uuid: "edf6d196a91e4256a94d85a8f82b4cc8" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43573 } }
I20260926 06:22:51.637782 19184 leader_election.cc:288] T 2e880990ab2a4dbcb874a48301b96ae4 P 5e2dd6c2c8e245038da7f222b8dd7850 [CANDIDATE]: Term 1 election: Requested vote from peers 42e3a388aa6a4797afbbc09559b9658f (127.4.202.129:46793), edf6d196a91e4256a94d85a8f82b4cc8 (127.4.202.130:43573)
I20260926 06:22:51.637949 18849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e880990ab2a4dbcb874a48301b96ae4" candidate_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42e3a388aa6a4797afbbc09559b9658f"
I20260926 06:22:51.638018 18983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e880990ab2a4dbcb874a48301b96ae4" candidate_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edf6d196a91e4256a94d85a8f82b4cc8"
I20260926 06:22:51.638041 18849 raft_consensus.cc:3037] T 2e880990ab2a4dbcb874a48301b96ae4 P 42e3a388aa6a4797afbbc09559b9658f [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:22:51.638095 18983 raft_consensus.cc:3037] T 2e880990ab2a4dbcb874a48301b96ae4 P edf6d196a91e4256a94d85a8f82b4cc8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:22:51.638857 18983 raft_consensus.cc:2454] T 2e880990ab2a4dbcb874a48301b96ae4 P edf6d196a91e4256a94d85a8f82b4cc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2dd6c2c8e245038da7f222b8dd7850 in term 1.
I20260926 06:22:51.638868 18849 raft_consensus.cc:2454] T 2e880990ab2a4dbcb874a48301b96ae4 P 42e3a388aa6a4797afbbc09559b9658f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2dd6c2c8e245038da7f222b8dd7850 in term 1.
I20260926 06:22:51.639019 19053 leader_election.cc:302] T 2e880990ab2a4dbcb874a48301b96ae4 P 5e2dd6c2c8e245038da7f222b8dd7850 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e2dd6c2c8e245038da7f222b8dd7850, edf6d196a91e4256a94d85a8f82b4cc8; no voters:
I20260926 06:22:51.639153 19184 raft_consensus.cc:2781] T 2e880990ab2a4dbcb874a48301b96ae4 P 5e2dd6c2c8e245038da7f222b8dd7850 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:22:51.639346 19184 raft_consensus.cc:686] T 2e880990ab2a4dbcb874a48301b96ae4 P 5e2dd6c2c8e245038da7f222b8dd7850 [term 1 LEADER]: Becoming Leader. State: Replica: 5e2dd6c2c8e245038da7f222b8dd7850, State: Running, Role: LEADER
I20260926 06:22:51.639442 19184 consensus_queue.cc:237] T 2e880990ab2a4dbcb874a48301b96ae4 P 5e2dd6c2c8e245038da7f222b8dd7850 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40851 } } peers { permanent_uuid: "42e3a388aa6a4797afbbc09559b9658f" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 46793 } } peers { permanent_uuid: "edf6d196a91e4256a94d85a8f82b4cc8" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43573 } }
I20260926 06:22:51.640434 18703 catalog_manager.cc:5495] T 2e880990ab2a4dbcb874a48301b96ae4 P 5e2dd6c2c8e245038da7f222b8dd7850 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e2dd6c2c8e245038da7f222b8dd7850 (127.4.202.131). New cstate: current_term: 1 leader_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42e3a388aa6a4797afbbc09559b9658f" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 46793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "edf6d196a91e4256a94d85a8f82b4cc8" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43573 } health_report { overall_health: UNKNOWN } } }
W20260926 06:22:51.642727 18896 tablet.cc:2369] T 2e880990ab2a4dbcb874a48301b96ae4 P 42e3a388aa6a4797afbbc09559b9658f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:22:51.657703 19173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:51.655500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44654 (local address 127.4.202.131:40851)
0926 06:22:51.655560 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:51.655561 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:51.655569 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:22:51.655635 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:51.655636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:51.655657 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:51.655734 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:22:51.655859 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.656317 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.657010 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.657180 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.657240 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:51.657391 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:22:51.657479 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:22:51.657538 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:51.657573 (+ 35us) server_negotiation.cc:299] Negotiation successful
0926 06:22:51.657588 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 06:22:51.659483 18849 raft_consensus.cc:1261] T 2e880990ab2a4dbcb874a48301b96ae4 P 42e3a388aa6a4797afbbc09559b9658f [term 1 FOLLOWER]: Refusing update from remote peer 5e2dd6c2c8e245038da7f222b8dd7850: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:22:51.659552 18983 raft_consensus.cc:1261] T 2e880990ab2a4dbcb874a48301b96ae4 P edf6d196a91e4256a94d85a8f82b4cc8 [term 1 FOLLOWER]: Refusing update from remote peer 5e2dd6c2c8e245038da7f222b8dd7850: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:22:51.659825 19187 consensus_queue.cc:1035] T 2e880990ab2a4dbcb874a48301b96ae4 P 5e2dd6c2c8e245038da7f222b8dd7850 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42e3a388aa6a4797afbbc09559b9658f" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 46793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:22:51.659953 19184 consensus_queue.cc:1035] T 2e880990ab2a4dbcb874a48301b96ae4 P 5e2dd6c2c8e245038da7f222b8dd7850 [LEADER]: Connected to new peer: Peer: permanent_uuid: "edf6d196a91e4256a94d85a8f82b4cc8" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:22:51.662904 19192 mvcc.cc:205] Tried to move back new op lower bound from 7333493848714539008 to 7333493848635834368. Current Snapshot: MvccSnapshot[applied={T|T < 7333493848714539008}]
I20260926 06:22:51.663025 19191 mvcc.cc:205] Tried to move back new op lower bound from 7333493848714539008 to 7333493848635834368. Current Snapshot: MvccSnapshot[applied={T|T < 7333493848714539008 or (T in {7333493848714539008})}]
I20260926 06:22:51.664070 18704 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:22:51.664178 18704 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:22:51.664937 18704 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f: Sending DeleteTablet for 3 replicas of tablet 2e880990ab2a4dbcb874a48301b96ae4
I20260926 06:22:51.665254 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 18766
I20260926 06:22:51.665326 18829 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e880990ab2a4dbcb874a48301b96ae4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:22:51 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:34218
I20260926 06:22:51.665325 18963 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e880990ab2a4dbcb874a48301b96ae4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:22:51 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:51636
I20260926 06:22:51.665509 19097 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e880990ab2a4dbcb874a48301b96ae4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:22:51 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:44646
I20260926 06:22:51.665870 19200 tablet_replica.cc:330] T 2e880990ab2a4dbcb874a48301b96ae4 P 5e2dd6c2c8e245038da7f222b8dd7850: stopping tablet replica
I20260926 06:22:51.665961 19200 raft_consensus.cc:2229] T 2e880990ab2a4dbcb874a48301b96ae4 P 5e2dd6c2c8e245038da7f222b8dd7850 [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:22:51.666049 19200 raft_consensus.cc:2258] T 2e880990ab2a4dbcb874a48301b96ae4 P 5e2dd6c2c8e245038da7f222b8dd7850 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:22:51.666266 19199 tablet_replica.cc:330] T 2e880990ab2a4dbcb874a48301b96ae4 P edf6d196a91e4256a94d85a8f82b4cc8: stopping tablet replica
I20260926 06:22:51.666402 19199 raft_consensus.cc:2229] T 2e880990ab2a4dbcb874a48301b96ae4 P edf6d196a91e4256a94d85a8f82b4cc8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:22:51.666457 19200 ts_tablet_manager.cc:1905] T 2e880990ab2a4dbcb874a48301b96ae4 P 5e2dd6c2c8e245038da7f222b8dd7850: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:22:51.666615 19199 raft_consensus.cc:2258] T 2e880990ab2a4dbcb874a48301b96ae4 P edf6d196a91e4256a94d85a8f82b4cc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:22:51.667102 19199 ts_tablet_manager.cc:1905] T 2e880990ab2a4dbcb874a48301b96ae4 P edf6d196a91e4256a94d85a8f82b4cc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:22:51.668479 19200 ts_tablet_manager.cc:1918] T 2e880990ab2a4dbcb874a48301b96ae4 P 5e2dd6c2c8e245038da7f222b8dd7850: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:22:51.668558 19200 log.cc:1196] T 2e880990ab2a4dbcb874a48301b96ae4 P 5e2dd6c2c8e245038da7f222b8dd7850: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/wal/wals/2e880990ab2a4dbcb874a48301b96ae4
I20260926 06:22:51.668630 19199 ts_tablet_manager.cc:1918] T 2e880990ab2a4dbcb874a48301b96ae4 P edf6d196a91e4256a94d85a8f82b4cc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:22:51.668700 19199 log.cc:1196] T 2e880990ab2a4dbcb874a48301b96ae4 P edf6d196a91e4256a94d85a8f82b4cc8: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/wal/wals/2e880990ab2a4dbcb874a48301b96ae4
I20260926 06:22:51.668843 19200 ts_tablet_manager.cc:1939] T 2e880990ab2a4dbcb874a48301b96ae4 P 5e2dd6c2c8e245038da7f222b8dd7850: Deleting consensus metadata
I20260926 06:22:51.668977 19199 ts_tablet_manager.cc:1939] T 2e880990ab2a4dbcb874a48301b96ae4 P edf6d196a91e4256a94d85a8f82b4cc8: Deleting consensus metadata
I20260926 06:22:51.669246 18690 catalog_manager.cc:4841] TS 5e2dd6c2c8e245038da7f222b8dd7850 (127.4.202.131:40851): tablet 2e880990ab2a4dbcb874a48301b96ae4 (table test-table [id=b725533c3bbf4f2dac5eb3c1ae7c07f6]) successfully deleted
I20260926 06:22:51.669440 18691 catalog_manager.cc:4841] TS edf6d196a91e4256a94d85a8f82b4cc8 (127.4.202.130:43573): tablet 2e880990ab2a4dbcb874a48301b96ae4 (table test-table [id=b725533c3bbf4f2dac5eb3c1ae7c07f6]) successfully deleted
W20260926 06:22:51.670217 18690 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.202.129:46793 (error 108)
I20260926 06:22:51.670428 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 18900
I20260926 06:22:51.671196 19174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:51.670538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:46793 (local address 127.0.0.1:34228)
0926 06:22:51.671116 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:51.671161 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.129:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":558}
W20260926 06:22:51.671327 18690 catalog_manager.cc:4584] TS 42e3a388aa6a4797afbbc09559b9658f (127.4.202.129:46793): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2e880990ab2a4dbcb874a48301b96ae4: Network error: Client connection negotiation failed: client connection to 127.4.202.129:46793: connect: Connection refused (error 111)
I20260926 06:22:51.678283 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 19034
I20260926 06:22:51.683048 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 18672
I20260926 06:22:51.687865 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:42879
--webserver_interface=127.4.202.190
--webserver_port=39647
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:34237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:42879
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:22:51.765766 19202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:22:51.765972 19202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:22:51.765995 19202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:22:51.767141 19202 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:22:51.767181 19202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:22:51.767196 19202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:22:51.767208 19202 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:22:51.767221 19202 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:22:51.768615 19202 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:34237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:42879
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:42879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=39647
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:22:51.768802 19202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:22:51.768953 19202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:22:51.777132 19208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:51.777132 19211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:51.777129 19209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:22:51.777197 19202 server_base.cc:1034] running on GCE node
I20260926 06:22:51.777679 19202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:22:51.777983 19202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:22:51.779134 19202 hybrid_clock.cc:648] HybridClock initialized: now 1790403771779122 us; error 28 us; skew 500 ppm
Sep 26 06:22:51 dist-test-slave-pd0t krb5kdc[18659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403771, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:22:51.781576 19202 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:22:51.782011 19202 webserver.cc:460] Webserver started at http://127.4.202.190:39647/ using document root <none> and password file <none>
I20260926 06:22:51.782267 19202 fs_manager.cc:359] Metadata directory not provided
I20260926 06:22:51.782321 19202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:22:51.783756 19202 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:22:51.784533 19218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:51.784727 19202 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:22:51.784817 19202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "43b852defc4e495187ed2c8cb041f43f"
format_stamp: "Formatted at 2026-09-26 06:22:50 on dist-test-slave-pd0t"
server_key: "e6277eec2f5b03889b4c1fa590e30de2"
server_key_iv: "e1463e74589144647f9100985fcf9009"
server_key_version: "encryptionkey@0"
I20260926 06:22:51.785166 19202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:22:51.796420 19202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:22:51.798519 19202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:22:51.798736 19202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:22:51.802697 19202 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:42879
I20260926 06:22:51.802755 19270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:42879 every 8 connection(s)
I20260926 06:22:51.803081 19202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:22:51.804499 19271 sys_catalog.cc:263] Verifying existing consensus state
I20260926 06:22:51.805044 19271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f: Bootstrap starting.
I20260926 06:22:51.812273 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 19202
I20260926 06:22:51.812608 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.129:46793
--local_ip_for_outbound_sockets=127.4.202.129
--tserver_master_addrs=127.4.202.190:42879
--webserver_port=46665
--webserver_interface=127.4.202.129
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:34237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:22:51.813228 19271 log.cc:824] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f: Log is configured to *not* fsync() on all Append() calls
I20260926 06:22:51.815248 19271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 06:22:51.815541 19271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f: Bootstrap complete.
I20260926 06:22:51.817785 19271 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b852defc4e495187ed2c8cb041f43f" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42879 } }
I20260926 06:22:51.818017 19271 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43b852defc4e495187ed2c8cb041f43f, State: Initialized, Role: FOLLOWER
I20260926 06:22:51.818190 19271 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b852defc4e495187ed2c8cb041f43f" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42879 } }
I20260926 06:22:51.818266 19271 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:22:51.818331 19271 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:22:51.818391 19271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [term 1 FOLLOWER]: Advancing to term 2
I20260926 06:22:51.819409 19271 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b852defc4e495187ed2c8cb041f43f" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42879 } }
I20260926 06:22:51.819528 19271 leader_election.cc:302] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43b852defc4e495187ed2c8cb041f43f; no voters:
I20260926 06:22:51.819721 19271 leader_election.cc:288] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 06:22:51.819753 19275 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [term 2 FOLLOWER]: Leader election won for term 2
I20260926 06:22:51.819883 19275 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [term 2 LEADER]: Becoming Leader. State: Replica: 43b852defc4e495187ed2c8cb041f43f, State: Running, Role: LEADER
I20260926 06:22:51.819993 19271 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:22:51.820034 19275 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b852defc4e495187ed2c8cb041f43f" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42879 } }
I20260926 06:22:51.820313 19275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43b852defc4e495187ed2c8cb041f43f. Latest consensus state: current_term: 2 leader_uuid: "43b852defc4e495187ed2c8cb041f43f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b852defc4e495187ed2c8cb041f43f" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42879 } } }
I20260926 06:22:51.820399 19275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [sys.catalog]: This master's current role is: LEADER
I20260926 06:22:51.820317 19276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "43b852defc4e495187ed2c8cb041f43f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b852defc4e495187ed2c8cb041f43f" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42879 } } }
I20260926 06:22:51.820588 19276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [sys.catalog]: This master's current role is: LEADER
I20260926 06:22:51.820601 19281 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:22:51.821570 19281 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:22:51.821687 19281 catalog_manager.cc:1259] Loaded cluster ID: ce89315171d34522a610eed6b3b403d5
I20260926 06:22:51.821807 19281 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:22:51.822309 19281 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260926 06:22:51.827412 19281 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260926 06:22:51.829707 19281 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:22:51.829888 19281 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f: Loaded TSK: 0
I20260926 06:22:51.830096 19281 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 06:22:51.895232 19274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:22:51.895407 19274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:22:51.895429 19274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:22:51.896508 19274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:22:51.896548 19274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:22:51.896574 19274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.129
I20260926 06:22:51.897969 19274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:34237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.129:46793
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.202.129
--webserver_port=46665
--tserver_master_addrs=127.4.202.190:42879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.129
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:22:51.898187 19274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:22:51.898419 19274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:22:51.904237 19298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:51.904256 19299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:51.904497 19301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:22:51.904348 19274 server_base.cc:1034] running on GCE node
I20260926 06:22:51.904742 19274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:22:51.905026 19274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:22:51.906184 19274 hybrid_clock.cc:648] HybridClock initialized: now 1790403771906170 us; error 30 us; skew 500 ppm
Sep 26 06:22:51 dist-test-slave-pd0t krb5kdc[18659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403771, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:22:51.908633 19274 init.cc:377] Logged in from keytab as kudu/127.4.202.129@KRBTEST.COM (short username kudu)
I20260926 06:22:51.909090 19274 webserver.cc:460] Webserver started at http://127.4.202.129:46665/ using document root <none> and password file <none>
I20260926 06:22:51.909330 19274 fs_manager.cc:359] Metadata directory not provided
I20260926 06:22:51.909387 19274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:22:51.910727 19274 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:22:51.911417 19308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:51.911562 19274 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:51.911644 19274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/wal
uuid: "42e3a388aa6a4797afbbc09559b9658f"
format_stamp: "Formatted at 2026-09-26 06:22:51 on dist-test-slave-pd0t"
server_key: "9f573e3bc6b4a7b6d5691861c6996f7f"
server_key_iv: "90b3ac96a9e0d75ff77cf2d53635340a"
server_key_version: "encryptionkey@0"
I20260926 06:22:51.911966 19274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:22:51.919389 19274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:22:51.921722 19274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:22:51.921871 19274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:22:51.922106 19274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:22:51.922525 19315 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 06:22:51.923894 19274 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 06:22:51.923941 19274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260926 06:22:51.923971 19274 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 06:22:51.924562 19274 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 06:22:51.924603 19274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260926 06:22:51.924643 19315 tablet_bootstrap.cc:492] T 2e880990ab2a4dbcb874a48301b96ae4 P 42e3a388aa6a4797afbbc09559b9658f: Bootstrap starting.
I20260926 06:22:51.932116 19274 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.129:46793
I20260926 06:22:51.932634 19274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
I20260926 06:22:51.933218 19422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.129:46793 every 8 connection(s)
I20260926 06:22:51.935729 19315 log.cc:824] T 2e880990ab2a4dbcb874a48301b96ae4 P 42e3a388aa6a4797afbbc09559b9658f: Log is configured to *not* fsync() on all Append() calls
I20260926 06:22:51.937719 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 19274
I20260926 06:22:51.938015 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.130:43573
--local_ip_for_outbound_sockets=127.4.202.130
--tserver_master_addrs=127.4.202.190:42879
--webserver_port=41525
--webserver_interface=127.4.202.130
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:34237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:22:51.938061 19315 tablet_bootstrap.cc:492] T 2e880990ab2a4dbcb874a48301b96ae4 P 42e3a388aa6a4797afbbc09559b9658f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 06:22:51.938315 19315 tablet_bootstrap.cc:492] T 2e880990ab2a4dbcb874a48301b96ae4 P 42e3a388aa6a4797afbbc09559b9658f: Bootstrap complete.
I20260926 06:22:51.939530 19315 ts_tablet_manager.cc:1397] T 2e880990ab2a4dbcb874a48301b96ae4 P 42e3a388aa6a4797afbbc09559b9658f: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.009s
I20260926 06:22:51.940841 19315 raft_consensus.cc:348] T 2e880990ab2a4dbcb874a48301b96ae4 P 42e3a388aa6a4797afbbc09559b9658f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40851 } } peers { permanent_uuid: "42e3a388aa6a4797afbbc09559b9658f" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 46793 } } peers { permanent_uuid: "edf6d196a91e4256a94d85a8f82b4cc8" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43573 } }
I20260926 06:22:51.941090 19315 raft_consensus.cc:729] T 2e880990ab2a4dbcb874a48301b96ae4 P 42e3a388aa6a4797afbbc09559b9658f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42e3a388aa6a4797afbbc09559b9658f, State: Initialized, Role: FOLLOWER
I20260926 06:22:51.941514 19315 consensus_queue.cc:260] T 2e880990ab2a4dbcb874a48301b96ae4 P 42e3a388aa6a4797afbbc09559b9658f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40851 } } peers { permanent_uuid: "42e3a388aa6a4797afbbc09559b9658f" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 46793 } } peers { permanent_uuid: "edf6d196a91e4256a94d85a8f82b4cc8" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43573 } }
I20260926 06:22:51.941811 19315 ts_tablet_manager.cc:1428] T 2e880990ab2a4dbcb874a48301b96ae4 P 42e3a388aa6a4797afbbc09559b9658f: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
Sep 26 06:22:51 dist-test-slave-pd0t krb5kdc[18659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403771, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:22:51.946810 19425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:51.933405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.129:50769)
0926 06:22:51.934689 (+ 1284us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:51.934737 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:51.935389 (+ 652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:51.936015 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:51.936018 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:51.936317 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:22:51.936567 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:51.936573 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.938552 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.938553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:51.939865 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:51.939868 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.940729 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.940730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:51.940904 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:51.941460 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:22:51.941469 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:22:51.943154 (+ 1685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:22:51.944886 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:51.944892 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:51.944894 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:51.945161 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:51.945468 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:51.945469 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:51.945470 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:51.945585 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:51.946156 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:22:51.946161 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:22:51.946309 (+ 148us) client_negotiation.cc:769] Sending connection context
0926 06:22:51.946426 (+ 117us) client_negotiation.cc:240] Negotiation successful
0926 06:22:51.946615 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1240,"thread_start_us":84,"threads_started":1}
I20260926 06:22:51.946967 19426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:51.933507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:50769 (local address 127.4.202.190:42879)
0926 06:22:51.933724 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:51.933736 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:51.934845 (+ 1109us) server_negotiation.cc:407] Connection header received
0926 06:22:51.935637 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:51.935645 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:51.935703 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:51.935865 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:51.936694 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.938423 (+ 1729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.939978 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.940606 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.940730 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:51.943266 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:51.943270 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:51.943270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:51.943279 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:51.944808 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:51.945272 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:51.945275 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:51.945276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:51.945401 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:51.945664 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:51.945665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:51.945666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:51.946080 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:51.946172 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:51.946558 (+ 386us) server_negotiation.cc:299] Negotiation successful
0926 06:22:51.946739 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":93,"threads_started":1}
I20260926 06:22:51.947464 19423 heartbeater.cc:347] Connected to a master server at 127.4.202.190:42879
I20260926 06:22:51.947551 19423 heartbeater.cc:464] Registering TS with master...
I20260926 06:22:51.947712 19423 heartbeater.cc:511] Master 127.4.202.190:42879 requested a full tablet report, sending...
I20260926 06:22:51.948339 19235 ts_manager.cc:194] Registered new tserver with Master: 42e3a388aa6a4797afbbc09559b9658f (127.4.202.129:46793)
I20260926 06:22:51.949493 19235 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.129@KRBTEST.COM'} at 127.4.202.129:50769
I20260926 06:22:51.950322 19423 heartbeater.cc:503] Master 127.4.202.190:42879 was elected leader, sending a full tablet report...
I20260926 06:22:51.954480 19430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:51.948954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:46793 (local address 127.0.0.1:34238)
0926 06:22:51.950723 (+ 1769us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:51.950766 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:51.950837 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:51.951166 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:51.951169 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:51.951171 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:22:51.951225 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:51.951226 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.952557 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.952558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:51.953340 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:51.953342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.954230 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.954231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:51.954332 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:51.954342 (+ 10us) client_negotiation.cc:769] Sending connection context
0926 06:22:51.954395 (+ 53us) client_negotiation.cc:240] Negotiation successful
0926 06:22:51.954430 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1733,"thread_start_us":81,"threads_started":1}
I20260926 06:22:51.954530 19431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:51.949198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34238 (local address 127.4.202.129:46793)
0926 06:22:51.950975 (+ 1777us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:51.950979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:51.950989 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:22:51.951025 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:51.951027 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:51.951043 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:51.951128 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:22:51.951307 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.952467 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.953442 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:51.954153 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:51.954226 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:51.954289 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:51.954427 (+ 138us) server_negotiation.cc:299] Negotiation successful
0926 06:22:51.954464 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1744,"thread_start_us":86,"threads_started":1}
I20260926 06:22:51.954926 19346 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e880990ab2a4dbcb874a48301b96ae4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:22:51 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:34238
I20260926 06:22:51.955386 19432 tablet_replica.cc:330] stopping tablet replica
I20260926 06:22:51.955484 19432 raft_consensus.cc:2229] T 2e880990ab2a4dbcb874a48301b96ae4 P 42e3a388aa6a4797afbbc09559b9658f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:22:51.955549 19432 raft_consensus.cc:2258] T 2e880990ab2a4dbcb874a48301b96ae4 P 42e3a388aa6a4797afbbc09559b9658f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:22:51.955849 19432 ts_tablet_manager.cc:1905] T 2e880990ab2a4dbcb874a48301b96ae4 P 42e3a388aa6a4797afbbc09559b9658f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:22:51.957901 19432 ts_tablet_manager.cc:1918] T 2e880990ab2a4dbcb874a48301b96ae4 P 42e3a388aa6a4797afbbc09559b9658f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:22:51.957960 19432 log.cc:1196] T 2e880990ab2a4dbcb874a48301b96ae4 P 42e3a388aa6a4797afbbc09559b9658f: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/wal/wals/2e880990ab2a4dbcb874a48301b96ae4
I20260926 06:22:51.958175 19432 ts_tablet_manager.cc:1939] T 2e880990ab2a4dbcb874a48301b96ae4 P 42e3a388aa6a4797afbbc09559b9658f: Deleting consensus metadata
I20260926 06:22:51.958688 19221 catalog_manager.cc:4841] TS 42e3a388aa6a4797afbbc09559b9658f (127.4.202.129:46793): tablet 2e880990ab2a4dbcb874a48301b96ae4 (table test-table [id=b725533c3bbf4f2dac5eb3c1ae7c07f6]) successfully deleted
W20260926 06:22:52.024899 19428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:22:52.025118 19428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:22:52.025159 19428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:22:52.026268 19428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:22:52.026335 19428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:22:52.026376 19428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.130
I20260926 06:22:52.027860 19428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:34237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.130:43573
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.202.130
--webserver_port=41525
--tserver_master_addrs=127.4.202.190:42879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.130
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:22:52.028117 19428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:22:52.028340 19428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:22:52.033090 19438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:52.033125 19437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:52.033094 19440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:22:52.033439 19428 server_base.cc:1034] running on GCE node
I20260926 06:22:52.033643 19428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:22:52.033877 19428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:22:52.035035 19428 hybrid_clock.cc:648] HybridClock initialized: now 1790403772035019 us; error 27 us; skew 500 ppm
Sep 26 06:22:52 dist-test-slave-pd0t krb5kdc[18659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403772, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:22:52.037520 19428 init.cc:377] Logged in from keytab as kudu/127.4.202.130@KRBTEST.COM (short username kudu)
I20260926 06:22:52.037954 19428 webserver.cc:460] Webserver started at http://127.4.202.130:41525/ using document root <none> and password file <none>
I20260926 06:22:52.038121 19428 fs_manager.cc:359] Metadata directory not provided
I20260926 06:22:52.038168 19428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:22:52.039459 19428 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:22:52.040004 19447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:52.040174 19428 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:22:52.040254 19428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/wal
uuid: "edf6d196a91e4256a94d85a8f82b4cc8"
format_stamp: "Formatted at 2026-09-26 06:22:51 on dist-test-slave-pd0t"
server_key: "41d2b5ca8edfb75534d0651171d8be90"
server_key_iv: "d722cea4dc482c485608577559f94195"
server_key_version: "encryptionkey@0"
I20260926 06:22:52.040541 19428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:22:52.047848 19428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:22:52.049655 19428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:22:52.049811 19428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:22:52.050035 19428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:22:52.050326 19428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:22:52.050382 19428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:52.050434 19428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:22:52.050463 19428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:52.056984 19428 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.130:43573
I20260926 06:22:52.057051 19560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.130:43573 every 8 connection(s)
I20260926 06:22:52.057399 19428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
I20260926 06:22:52.057698 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 19428
I20260926 06:22:52.057961 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.131:40851
--local_ip_for_outbound_sockets=127.4.202.131
--tserver_master_addrs=127.4.202.190:42879
--webserver_port=44251
--webserver_interface=127.4.202.131
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:34237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 06:22:52 dist-test-slave-pd0t krb5kdc[18659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403772, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:22:52.067842 19426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:52.058413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:48717 (local address 127.4.202.190:42879)
0926 06:22:52.058661 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:52.058663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:52.058673 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:22:52.059257 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:52.059259 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:52.059274 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:52.059344 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:52.060321 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:52.060934 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:52.062371 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:52.062568 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:52.062670 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:52.065001 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:52.065003 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:52.065003 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:52.065011 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:52.066459 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:52.066865 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:52.066866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:52.066866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:52.066942 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:52.067187 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:52.067188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:52.067189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:52.067298 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:52.067375 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:52.067655 (+ 280us) server_negotiation.cc:299] Negotiation successful
0926 06:22:52.067740 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227}
I20260926 06:22:52.068055 19563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:52.058154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.130:48717)
0926 06:22:52.058346 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:52.058393 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:52.059144 (+ 751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:52.059441 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:52.059444 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:52.060057 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:22:52.060229 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:52.060234 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:52.061017 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:52.061018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:52.062292 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:52.062296 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:52.062690 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:52.062691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:52.062853 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:52.063330 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:22:52.063340 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:22:52.064906 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:22:52.066553 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:52.066561 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:52.066562 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:52.066792 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:52.067010 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:52.067011 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:52.067012 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:52.067120 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:52.067367 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:22:52.067373 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:22:52.067525 (+ 152us) client_negotiation.cc:769] Sending connection context
0926 06:22:52.067633 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 06:22:52.067786 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":85,"threads_started":1}
I20260926 06:22:52.068617 19561 heartbeater.cc:347] Connected to a master server at 127.4.202.190:42879
I20260926 06:22:52.068724 19561 heartbeater.cc:464] Registering TS with master...
I20260926 06:22:52.068903 19561 heartbeater.cc:511] Master 127.4.202.190:42879 requested a full tablet report, sending...
I20260926 06:22:52.069321 19235 ts_manager.cc:194] Registered new tserver with Master: edf6d196a91e4256a94d85a8f82b4cc8 (127.4.202.130:43573)
I20260926 06:22:52.069810 19235 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.130@KRBTEST.COM'} at 127.4.202.130:48717
W20260926 06:22:52.139480 19564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:22:52.139662 19564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:22:52.139683 19564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:22:52.140707 19564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:22:52.140736 19564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:22:52.140760 19564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.131
I20260926 06:22:52.142035 19564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:34237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.131:40851
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.202.131
--webserver_port=44251
--tserver_master_addrs=127.4.202.190:42879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.131
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:22:52.142208 19564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:22:52.142380 19564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:22:52.147277 19569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:52.147274 19570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:52.147324 19572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:22:52.147526 19564 server_base.cc:1034] running on GCE node
I20260926 06:22:52.147744 19564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:22:52.147987 19564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:22:52.149148 19564 hybrid_clock.cc:648] HybridClock initialized: now 1790403772149134 us; error 27 us; skew 500 ppm
Sep 26 06:22:52 dist-test-slave-pd0t krb5kdc[18659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403772, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:22:52.151540 19564 init.cc:377] Logged in from keytab as kudu/127.4.202.131@KRBTEST.COM (short username kudu)
I20260926 06:22:52.151973 19564 webserver.cc:460] Webserver started at http://127.4.202.131:44251/ using document root <none> and password file <none>
I20260926 06:22:52.152170 19564 fs_manager.cc:359] Metadata directory not provided
I20260926 06:22:52.152253 19564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:22:52.153535 19564 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:22:52.154103 19579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:52.154269 19564 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:22:52.154407 19564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/wal
uuid: "5e2dd6c2c8e245038da7f222b8dd7850"
format_stamp: "Formatted at 2026-09-26 06:22:51 on dist-test-slave-pd0t"
server_key: "6e6ba8d54dc704ef4de3bb13e5750299"
server_key_iv: "df546a04578b5850d57da5890577398e"
server_key_version: "encryptionkey@0"
I20260926 06:22:52.154764 19564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:22:52.164155 19564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:22:52.165963 19564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:22:52.166110 19564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:22:52.166318 19564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:22:52.166651 19564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:22:52.166692 19564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:52.166728 19564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:22:52.166757 19564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:52.173131 19564 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.131:40851
I20260926 06:22:52.173164 19692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.131:40851 every 8 connection(s)
I20260926 06:22:52.173514 19564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
Sep 26 06:22:52 dist-test-slave-pd0t krb5kdc[18659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403772, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:22:52.182080 19426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:52.174223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:36911 (local address 127.4.202.190:42879)
0926 06:22:52.174270 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:52.174272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:52.174481 (+ 209us) server_negotiation.cc:407] Connection header received
0926 06:22:52.174982 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:52.174983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:52.174993 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:52.175058 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:52.175997 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:52.176499 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:52.177446 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:52.177624 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:52.177673 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:52.179678 (+ 2005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:52.179680 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:52.179680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:52.179686 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:52.180923 (+ 1237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:52.181335 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:52.181336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:52.181336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:52.181392 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:52.181603 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:52.181603 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:52.181604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:52.181699 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:52.181748 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:52.181954 (+ 206us) server_negotiation.cc:299] Negotiation successful
0926 06:22:52.182021 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:22:52.182192 19695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:52.174195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.131:36911)
0926 06:22:52.174372 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:52.174410 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:52.174892 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:52.175128 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:52.175131 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:52.175715 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:22:52.175888 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:52.175893 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:52.176590 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:52.176591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:52.177370 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:52.177372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:52.177694 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:52.177694 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:52.177788 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:52.178209 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:22:52.178216 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:22:52.179594 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:22:52.180991 (+ 1397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:52.180997 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:52.180998 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:52.181199 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:52.181457 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:52.181458 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:52.181458 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:52.181547 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:52.181752 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:22:52.181756 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:22:52.181852 (+ 96us) client_negotiation.cc:769] Sending connection context
0926 06:22:52.181926 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 06:22:52.182028 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":109,"threads_started":1}
I20260926 06:22:52.182749 19693 heartbeater.cc:347] Connected to a master server at 127.4.202.190:42879
I20260926 06:22:52.182830 19693 heartbeater.cc:464] Registering TS with master...
I20260926 06:22:52.183013 19693 heartbeater.cc:511] Master 127.4.202.190:42879 requested a full tablet report, sending...
I20260926 06:22:52.183110 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 19564
I20260926 06:22:52.183494 19235 ts_manager.cc:194] Registered new tserver with Master: 5e2dd6c2c8e245038da7f222b8dd7850 (127.4.202.131:40851)
I20260926 06:22:52.183998 19235 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.131@KRBTEST.COM'} at 127.4.202.131:36911
I20260926 06:22:52.189327 19426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:52.183535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49210 (local address 127.4.202.190:42879)
0926 06:22:52.183605 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:52.183607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:52.183707 (+ 100us) server_negotiation.cc:407] Connection header received
0926 06:22:52.183733 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:52.183734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:52.183744 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:52.183815 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:52.184418 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:52.184873 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:52.185545 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:52.185729 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:52.185786 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:52.186983 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:52.186984 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:52.186984 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:52.186990 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:52.188334 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:52.188709 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:52.188709 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:52.188710 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:52.188758 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:52.188964 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:52.188965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:52.188965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:52.189052 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:52.189159 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:52.189212 (+ 53us) server_negotiation.cc:299] Negotiation successful
0926 06:22:52.189261 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 06:22:52.190140 4906 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:22:52.192539 19426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:52.190553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49218 (local address 127.4.202.190:42879)
0926 06:22:52.190610 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:52.190612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:52.190620 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:22:52.190685 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:52.190686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:52.190712 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:52.190777 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:22:52.190847 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:52.191284 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:52.191944 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:52.192118 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:52.192166 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:52.192312 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:22:52.192389 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:22:52.192444 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:52.192488 (+ 44us) server_negotiation.cc:299] Negotiation successful
0926 06:22:52.192500 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 06:22:52.192777 19235 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:49218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:22:52.194218 19235 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:22:52.196463 19346 tablet_service.cc:1467] Processing CreateTablet for tablet 6f3f8b7fcdfb406c952d54ac848b382c (DEFAULT_TABLE table=test-table [id=ebeb60ce4f6642d085b2b20cb7f4f81e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:22:52.196650 19346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f3f8b7fcdfb406c952d54ac848b382c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:22:52.198052 19315 tablet_bootstrap.cc:492] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f: Bootstrap starting.
I20260926 06:22:52.198379 19315 tablet_bootstrap.cc:654] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f: Neither blocks nor log segments found. Creating new log.
I20260926 06:22:52.198982 19315 tablet_bootstrap.cc:492] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f: No bootstrap required, opened a new log
I20260926 06:22:52.199029 19315 ts_tablet_manager.cc:1397] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260926 06:22:52.199119 19315 raft_consensus.cc:348] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf6d196a91e4256a94d85a8f82b4cc8" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43573 } } peers { permanent_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40851 } } peers { permanent_uuid: "42e3a388aa6a4797afbbc09559b9658f" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 46793 } }
I20260926 06:22:52.199189 19315 raft_consensus.cc:374] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:22:52.199234 19315 raft_consensus.cc:729] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42e3a388aa6a4797afbbc09559b9658f, State: Initialized, Role: FOLLOWER
I20260926 06:22:52.199298 19315 consensus_queue.cc:260] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf6d196a91e4256a94d85a8f82b4cc8" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43573 } } peers { permanent_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40851 } } peers { permanent_uuid: "42e3a388aa6a4797afbbc09559b9658f" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 46793 } }
I20260926 06:22:52.199385 19315 ts_tablet_manager.cc:1428] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:52.199958 19430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:52.196029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:43573 (local address 127.0.0.1:51642)
0926 06:22:52.196081 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:52.196090 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:52.196129 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:52.196529 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:52.196530 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:52.196531 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:22:52.196566 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:52.196567 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:52.197848 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:52.197849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:52.198828 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:52.198831 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:52.199739 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:52.199740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:52.199811 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:52.199815 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:22:52.199844 (+ 29us) client_negotiation.cc:240] Negotiation successful
0926 06:22:52.199888 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260926 06:22:52.200304 19699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:52.196130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51642 (local address 127.4.202.130:43573)
0926 06:22:52.196374 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:52.196379 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:52.196388 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:22:52.196428 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:52.196430 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:52.196447 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:52.196521 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:22:52.196616 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:52.197730 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:52.198900 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:52.199644 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:52.200058 (+ 414us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:52.200148 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:52.200172 (+ 24us) server_negotiation.cc:299] Negotiation successful
0926 06:22:52.200211 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":82,"threads_started":1}
I20260926 06:22:52.200381 19698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:52.196084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:40851 (local address 127.0.0.1:44668)
0926 06:22:52.196198 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:52.196206 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:52.196256 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:52.196727 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:52.196729 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:52.196731 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:22:52.196771 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:52.196772 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:52.197973 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:52.197974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:52.199113 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:52.199117 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:52.200148 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:52.200148 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:52.200207 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:52.200211 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:22:52.200263 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 06:22:52.200307 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":67,"threads_started":1}
I20260926 06:22:52.200624 19700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:52.196365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44668 (local address 127.4.202.131:40851)
0926 06:22:52.196526 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:52.196530 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:52.196541 (+ 11us) server_negotiation.cc:407] Connection header received
0926 06:22:52.196574 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:52.196576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:52.196589 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:52.196668 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:22:52.196850 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:52.197869 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:52.199208 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:52.200065 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:52.200140 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:52.200193 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:52.200513 (+ 320us) server_negotiation.cc:299] Negotiation successful
0926 06:22:52.200556 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":81,"threads_started":1}
I20260926 06:22:52.200911 19495 tablet_service.cc:1467] Processing CreateTablet for tablet 6f3f8b7fcdfb406c952d54ac848b382c (DEFAULT_TABLE table=test-table [id=ebeb60ce4f6642d085b2b20cb7f4f81e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:22:52.201205 19495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f3f8b7fcdfb406c952d54ac848b382c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:22:52.203778 19702 tablet_bootstrap.cc:492] T 6f3f8b7fcdfb406c952d54ac848b382c P edf6d196a91e4256a94d85a8f82b4cc8: Bootstrap starting.
I20260926 06:22:52.204506 19702 tablet_bootstrap.cc:654] T 6f3f8b7fcdfb406c952d54ac848b382c P edf6d196a91e4256a94d85a8f82b4cc8: Neither blocks nor log segments found. Creating new log.
I20260926 06:22:52.204330 19627 tablet_service.cc:1467] Processing CreateTablet for tablet 6f3f8b7fcdfb406c952d54ac848b382c (DEFAULT_TABLE table=test-table [id=ebeb60ce4f6642d085b2b20cb7f4f81e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:22:52.204738 19627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f3f8b7fcdfb406c952d54ac848b382c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:22:52.205016 19702 log.cc:824] T 6f3f8b7fcdfb406c952d54ac848b382c P edf6d196a91e4256a94d85a8f82b4cc8: Log is configured to *not* fsync() on all Append() calls
I20260926 06:22:52.205703 19702 tablet_bootstrap.cc:492] T 6f3f8b7fcdfb406c952d54ac848b382c P edf6d196a91e4256a94d85a8f82b4cc8: No bootstrap required, opened a new log
I20260926 06:22:52.205799 19702 ts_tablet_manager.cc:1397] T 6f3f8b7fcdfb406c952d54ac848b382c P edf6d196a91e4256a94d85a8f82b4cc8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:22:52.207110 19702 raft_consensus.cc:348] T 6f3f8b7fcdfb406c952d54ac848b382c P edf6d196a91e4256a94d85a8f82b4cc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf6d196a91e4256a94d85a8f82b4cc8" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43573 } } peers { permanent_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40851 } } peers { permanent_uuid: "42e3a388aa6a4797afbbc09559b9658f" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 46793 } }
I20260926 06:22:52.207263 19702 raft_consensus.cc:374] T 6f3f8b7fcdfb406c952d54ac848b382c P edf6d196a91e4256a94d85a8f82b4cc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:22:52.207329 19702 raft_consensus.cc:729] T 6f3f8b7fcdfb406c952d54ac848b382c P edf6d196a91e4256a94d85a8f82b4cc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edf6d196a91e4256a94d85a8f82b4cc8, State: Initialized, Role: FOLLOWER
I20260926 06:22:52.207495 19702 consensus_queue.cc:260] T 6f3f8b7fcdfb406c952d54ac848b382c P edf6d196a91e4256a94d85a8f82b4cc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf6d196a91e4256a94d85a8f82b4cc8" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43573 } } peers { permanent_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40851 } } peers { permanent_uuid: "42e3a388aa6a4797afbbc09559b9658f" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 46793 } }
I20260926 06:22:52.207573 19704 tablet_bootstrap.cc:492] T 6f3f8b7fcdfb406c952d54ac848b382c P 5e2dd6c2c8e245038da7f222b8dd7850: Bootstrap starting.
I20260926 06:22:52.207777 19702 ts_tablet_manager.cc:1428] T 6f3f8b7fcdfb406c952d54ac848b382c P edf6d196a91e4256a94d85a8f82b4cc8: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 06:22:52.207926 19561 heartbeater.cc:503] Master 127.4.202.190:42879 was elected leader, sending a full tablet report...
I20260926 06:22:52.208230 19704 tablet_bootstrap.cc:654] T 6f3f8b7fcdfb406c952d54ac848b382c P 5e2dd6c2c8e245038da7f222b8dd7850: Neither blocks nor log segments found. Creating new log.
I20260926 06:22:52.208838 19704 log.cc:824] T 6f3f8b7fcdfb406c952d54ac848b382c P 5e2dd6c2c8e245038da7f222b8dd7850: Log is configured to *not* fsync() on all Append() calls
I20260926 06:22:52.209511 19704 tablet_bootstrap.cc:492] T 6f3f8b7fcdfb406c952d54ac848b382c P 5e2dd6c2c8e245038da7f222b8dd7850: No bootstrap required, opened a new log
I20260926 06:22:52.209594 19704 ts_tablet_manager.cc:1397] T 6f3f8b7fcdfb406c952d54ac848b382c P 5e2dd6c2c8e245038da7f222b8dd7850: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:22:52.210889 19704 raft_consensus.cc:348] T 6f3f8b7fcdfb406c952d54ac848b382c P 5e2dd6c2c8e245038da7f222b8dd7850 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf6d196a91e4256a94d85a8f82b4cc8" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43573 } } peers { permanent_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40851 } } peers { permanent_uuid: "42e3a388aa6a4797afbbc09559b9658f" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 46793 } }
I20260926 06:22:52.211033 19704 raft_consensus.cc:374] T 6f3f8b7fcdfb406c952d54ac848b382c P 5e2dd6c2c8e245038da7f222b8dd7850 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:22:52.211076 19704 raft_consensus.cc:729] T 6f3f8b7fcdfb406c952d54ac848b382c P 5e2dd6c2c8e245038da7f222b8dd7850 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2dd6c2c8e245038da7f222b8dd7850, State: Initialized, Role: FOLLOWER
I20260926 06:22:52.211202 19704 consensus_queue.cc:260] T 6f3f8b7fcdfb406c952d54ac848b382c P 5e2dd6c2c8e245038da7f222b8dd7850 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf6d196a91e4256a94d85a8f82b4cc8" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43573 } } peers { permanent_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40851 } } peers { permanent_uuid: "42e3a388aa6a4797afbbc09559b9658f" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 46793 } }
I20260926 06:22:52.211438 19704 ts_tablet_manager.cc:1428] T 6f3f8b7fcdfb406c952d54ac848b382c P 5e2dd6c2c8e245038da7f222b8dd7850: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:22:52.211510 19693 heartbeater.cc:503] Master 127.4.202.190:42879 was elected leader, sending a full tablet report...
W20260926 06:22:52.308058 19562 tablet.cc:2369] T 6f3f8b7fcdfb406c952d54ac848b382c P edf6d196a91e4256a94d85a8f82b4cc8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:22:52.355093 19429 raft_consensus.cc:482] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:22:52.355223 19429 raft_consensus.cc:504] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf6d196a91e4256a94d85a8f82b4cc8" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43573 } } peers { permanent_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40851 } } peers { permanent_uuid: "42e3a388aa6a4797afbbc09559b9658f" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 46793 } }
I20260926 06:22:52.355515 19429 leader_election.cc:288] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers edf6d196a91e4256a94d85a8f82b4cc8 (127.4.202.130:43573), 5e2dd6c2c8e245038da7f222b8dd7850 (127.4.202.131:40851)
I20260926 06:22:52.358920 19425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:52.355704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:40851 (local address 127.4.202.129:42253)
0926 06:22:52.355765 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:52.355779 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:52.355839 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:52.356031 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:52.356032 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:52.356034 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:22:52.356094 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:52.356096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:52.356817 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:52.356818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:52.358027 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:52.358030 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:52.358717 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:52.358718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:52.358769 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:52.358772 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:22:52.358827 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 06:22:52.358858 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260926 06:22:52.358952 19700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:52.355889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:42253 (local address 127.4.202.131:40851)
0926 06:22:52.355932 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:52.355936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:52.355948 (+ 12us) server_negotiation.cc:407] Connection header received
0926 06:22:52.355971 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:52.355972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:52.355979 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:52.356019 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:22:52.356190 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:52.356736 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:52.358128 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:52.358645 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:52.358715 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:52.358722 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:52.358850 (+ 128us) server_negotiation.cc:299] Negotiation successful
0926 06:22:52.358886 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 06:22:52.359352 19708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:52.355706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:43573 (local address 127.4.202.129:42903)
0926 06:22:52.355896 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:52.355908 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:52.355993 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:52.356160 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:52.356161 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:52.356162 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:22:52.356507 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:52.356509 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:52.357323 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:52.357324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:52.358341 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:52.358343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:52.359080 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:52.359081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:52.359148 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:52.359152 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:22:52.359224 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 06:22:52.359274 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":14,"thread_start_us":133,"threads_started":1}
I20260926 06:22:52.359378 19699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:52.355782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:42903 (local address 127.4.202.130:43573)
0926 06:22:52.355832 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:52.355834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:52.355978 (+ 144us) server_negotiation.cc:407] Connection header received
0926 06:22:52.356075 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:52.356076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:52.356085 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:52.356137 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:22:52.356592 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:52.357221 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:52.358415 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:52.358968 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:52.359035 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:52.359045 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:52.359263 (+ 218us) server_negotiation.cc:299] Negotiation successful
0926 06:22:52.359308 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 06:22:52.359335 19647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f3f8b7fcdfb406c952d54ac848b382c" candidate_uuid: "42e3a388aa6a4797afbbc09559b9658f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" is_pre_election: true
I20260926 06:22:52.359488 19647 raft_consensus.cc:2454] T 6f3f8b7fcdfb406c952d54ac848b382c P 5e2dd6c2c8e245038da7f222b8dd7850 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42e3a388aa6a4797afbbc09559b9658f in term 0.
I20260926 06:22:52.359707 19311 leader_election.cc:302] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42e3a388aa6a4797afbbc09559b9658f, 5e2dd6c2c8e245038da7f222b8dd7850; no voters:
I20260926 06:22:52.359747 19515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f3f8b7fcdfb406c952d54ac848b382c" candidate_uuid: "42e3a388aa6a4797afbbc09559b9658f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edf6d196a91e4256a94d85a8f82b4cc8" is_pre_election: true
I20260926 06:22:52.359838 19429 raft_consensus.cc:2781] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:22:52.359913 19429 raft_consensus.cc:482] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:22:52.359923 19515 raft_consensus.cc:2454] T 6f3f8b7fcdfb406c952d54ac848b382c P edf6d196a91e4256a94d85a8f82b4cc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42e3a388aa6a4797afbbc09559b9658f in term 0.
I20260926 06:22:52.359941 19429 raft_consensus.cc:3037] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:22:52.360780 19429 raft_consensus.cc:504] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf6d196a91e4256a94d85a8f82b4cc8" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43573 } } peers { permanent_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40851 } } peers { permanent_uuid: "42e3a388aa6a4797afbbc09559b9658f" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 46793 } }
I20260926 06:22:52.360926 19429 leader_election.cc:288] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [CANDIDATE]: Term 1 election: Requested vote from peers edf6d196a91e4256a94d85a8f82b4cc8 (127.4.202.130:43573), 5e2dd6c2c8e245038da7f222b8dd7850 (127.4.202.131:40851)
I20260926 06:22:52.361074 19515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f3f8b7fcdfb406c952d54ac848b382c" candidate_uuid: "42e3a388aa6a4797afbbc09559b9658f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edf6d196a91e4256a94d85a8f82b4cc8"
I20260926 06:22:52.361092 19647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f3f8b7fcdfb406c952d54ac848b382c" candidate_uuid: "42e3a388aa6a4797afbbc09559b9658f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2dd6c2c8e245038da7f222b8dd7850"
I20260926 06:22:52.361148 19515 raft_consensus.cc:3037] T 6f3f8b7fcdfb406c952d54ac848b382c P edf6d196a91e4256a94d85a8f82b4cc8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:22:52.361191 19647 raft_consensus.cc:3037] T 6f3f8b7fcdfb406c952d54ac848b382c P 5e2dd6c2c8e245038da7f222b8dd7850 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:22:52.361864 19515 raft_consensus.cc:2454] T 6f3f8b7fcdfb406c952d54ac848b382c P edf6d196a91e4256a94d85a8f82b4cc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42e3a388aa6a4797afbbc09559b9658f in term 1.
I20260926 06:22:52.361887 19647 raft_consensus.cc:2454] T 6f3f8b7fcdfb406c952d54ac848b382c P 5e2dd6c2c8e245038da7f222b8dd7850 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42e3a388aa6a4797afbbc09559b9658f in term 1.
I20260926 06:22:52.362046 19312 leader_election.cc:302] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42e3a388aa6a4797afbbc09559b9658f, edf6d196a91e4256a94d85a8f82b4cc8; no voters:
I20260926 06:22:52.362145 19429 raft_consensus.cc:2781] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:22:52.362313 19429 raft_consensus.cc:686] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [term 1 LEADER]: Becoming Leader. State: Replica: 42e3a388aa6a4797afbbc09559b9658f, State: Running, Role: LEADER
I20260926 06:22:52.362401 19429 consensus_queue.cc:237] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf6d196a91e4256a94d85a8f82b4cc8" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43573 } } peers { permanent_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40851 } } peers { permanent_uuid: "42e3a388aa6a4797afbbc09559b9658f" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 46793 } }
I20260926 06:22:52.363200 19235 catalog_manager.cc:5495] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f reported cstate change: term changed from 0 to 1, leader changed from <none> to 42e3a388aa6a4797afbbc09559b9658f (127.4.202.129). New cstate: current_term: 1 leader_uuid: "42e3a388aa6a4797afbbc09559b9658f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf6d196a91e4256a94d85a8f82b4cc8" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42e3a388aa6a4797afbbc09559b9658f" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 46793 } health_report { overall_health: HEALTHY } } }
I20260926 06:22:52.380272 19431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:52.378044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34244 (local address 127.4.202.129:46793)
0926 06:22:52.378109 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:52.378116 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:52.378124 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:22:52.378155 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:52.378156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:52.378172 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:52.378237 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:22:52.378380 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:52.378910 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:52.379632 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:52.379815 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:52.379864 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:52.380015 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:22:52.380107 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:22:52.380157 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:52.380200 (+ 43us) server_negotiation.cc:299] Negotiation successful
0926 06:22:52.380218 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 06:22:52.381901 19647 raft_consensus.cc:1261] T 6f3f8b7fcdfb406c952d54ac848b382c P 5e2dd6c2c8e245038da7f222b8dd7850 [term 1 FOLLOWER]: Refusing update from remote peer 42e3a388aa6a4797afbbc09559b9658f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:22:52.381896 19515 raft_consensus.cc:1261] T 6f3f8b7fcdfb406c952d54ac848b382c P edf6d196a91e4256a94d85a8f82b4cc8 [term 1 FOLLOWER]: Refusing update from remote peer 42e3a388aa6a4797afbbc09559b9658f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:22:52.382229 19429 consensus_queue.cc:1035] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:22:52.382364 19709 consensus_queue.cc:1035] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [LEADER]: Connected to new peer: Peer: permanent_uuid: "edf6d196a91e4256a94d85a8f82b4cc8" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:22:52.384897 19713 mvcc.cc:205] Tried to move back new op lower bound from 7333493851673358336 to 7333493851596894208. Current Snapshot: MvccSnapshot[applied={T|T < 7333493851673358336}]
I20260926 06:22:52.385228 19714 mvcc.cc:205] Tried to move back new op lower bound from 7333493851673358336 to 7333493851596894208. Current Snapshot: MvccSnapshot[applied={T|T < 7333493851673358336 or (T in {7333493851673358336})}]
I20260926 06:22:52.387231 19234 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:49218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:22:52.387336 19234 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:49218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:22:52.388103 19234 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f: Sending DeleteTablet for 3 replicas of tablet 6f3f8b7fcdfb406c952d54ac848b382c
I20260926 06:22:52.388396 19495 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f3f8b7fcdfb406c952d54ac848b382c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:22:52 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:51642
I20260926 06:22:52.388453 19346 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f3f8b7fcdfb406c952d54ac848b382c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:22:52 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:34238
I20260926 06:22:52.388521 19627 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f3f8b7fcdfb406c952d54ac848b382c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:22:52 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:44668
I20260926 06:22:52.388584 19432 tablet_replica.cc:330] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f: stopping tablet replica
I20260926 06:22:52.388620 19722 tablet_replica.cc:330] T 6f3f8b7fcdfb406c952d54ac848b382c P edf6d196a91e4256a94d85a8f82b4cc8: stopping tablet replica
I20260926 06:22:52.388644 19432 raft_consensus.cc:2229] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:22:52.388731 19722 raft_consensus.cc:2229] T 6f3f8b7fcdfb406c952d54ac848b382c P edf6d196a91e4256a94d85a8f82b4cc8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:22:52.388754 19432 raft_consensus.cc:2258] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:22:52.388827 19722 raft_consensus.cc:2258] T 6f3f8b7fcdfb406c952d54ac848b382c P edf6d196a91e4256a94d85a8f82b4cc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:22:52.388870 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 19274
I20260926 06:22:52.389736 19722 ts_tablet_manager.cc:1905] T 6f3f8b7fcdfb406c952d54ac848b382c P edf6d196a91e4256a94d85a8f82b4cc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:22:52.390012 19723 tablet_replica.cc:330] T 6f3f8b7fcdfb406c952d54ac848b382c P 5e2dd6c2c8e245038da7f222b8dd7850: stopping tablet replica
I20260926 06:22:52.390177 19723 raft_consensus.cc:2229] T 6f3f8b7fcdfb406c952d54ac848b382c P 5e2dd6c2c8e245038da7f222b8dd7850 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:22:52.390339 19723 raft_consensus.cc:2258] T 6f3f8b7fcdfb406c952d54ac848b382c P 5e2dd6c2c8e245038da7f222b8dd7850 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:22:52.390945 19723 ts_tablet_manager.cc:1905] T 6f3f8b7fcdfb406c952d54ac848b382c P 5e2dd6c2c8e245038da7f222b8dd7850: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:22:52.391242 19722 ts_tablet_manager.cc:1918] T 6f3f8b7fcdfb406c952d54ac848b382c P edf6d196a91e4256a94d85a8f82b4cc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:22:52.391328 19722 log.cc:1196] T 6f3f8b7fcdfb406c952d54ac848b382c P edf6d196a91e4256a94d85a8f82b4cc8: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/wal/wals/6f3f8b7fcdfb406c952d54ac848b382c
I20260926 06:22:52.391604 19722 ts_tablet_manager.cc:1939] T 6f3f8b7fcdfb406c952d54ac848b382c P edf6d196a91e4256a94d85a8f82b4cc8: Deleting consensus metadata
I20260926 06:22:52.392320 19222 catalog_manager.cc:4841] TS edf6d196a91e4256a94d85a8f82b4cc8 (127.4.202.130:43573): tablet 6f3f8b7fcdfb406c952d54ac848b382c (table test-table [id=ebeb60ce4f6642d085b2b20cb7f4f81e]) successfully deleted
I20260926 06:22:52.392758 19723 ts_tablet_manager.cc:1918] T 6f3f8b7fcdfb406c952d54ac848b382c P 5e2dd6c2c8e245038da7f222b8dd7850: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:22:52.392813 19723 log.cc:1196] T 6f3f8b7fcdfb406c952d54ac848b382c P 5e2dd6c2c8e245038da7f222b8dd7850: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/wal/wals/6f3f8b7fcdfb406c952d54ac848b382c
I20260926 06:22:52.393070 19723 ts_tablet_manager.cc:1939] T 6f3f8b7fcdfb406c952d54ac848b382c P 5e2dd6c2c8e245038da7f222b8dd7850: Deleting consensus metadata
I20260926 06:22:52.393544 19221 catalog_manager.cc:4841] TS 5e2dd6c2c8e245038da7f222b8dd7850 (127.4.202.131:40851): tablet 6f3f8b7fcdfb406c952d54ac848b382c (table test-table [id=ebeb60ce4f6642d085b2b20cb7f4f81e]) successfully deleted
W20260926 06:22:52.394800 19221 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.202.129:46793 (error 108)
I20260926 06:22:52.395066 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 19428
I20260926 06:22:52.395229 19698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:52.395100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:46793 (local address 127.0.0.1:34258)
0926 06:22:52.395159 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:52.395194 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.129:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260926 06:22:52.395946 19221 catalog_manager.cc:4584] TS 42e3a388aa6a4797afbbc09559b9658f (127.4.202.129:46793): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6f3f8b7fcdfb406c952d54ac848b382c: Network error: Client connection negotiation failed: client connection to 127.4.202.129:46793: connect: Connection refused (error 111)
I20260926 06:22:52.399617 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 19564
I20260926 06:22:52.404449 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 19202
I20260926 06:22:52.409725 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:42879
--webserver_interface=127.4.202.190
--webserver_port=39647
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:34237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:42879
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:22:52.489121 19725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:22:52.489318 19725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:22:52.489341 19725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:22:52.490487 19725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:22:52.490527 19725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:22:52.490542 19725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:22:52.490556 19725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:22:52.490568 19725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:22:52.492051 19725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:34237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:42879
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:42879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=39647
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:22:52.492303 19725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:22:52.492535 19725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 06:22:52.497305 19725 server_base.cc:1034] running on GCE node
W20260926 06:22:52.497285 19731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:52.497349 19733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:52.497288 19730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:22:52.497743 19725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:22:52.498037 19725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:22:52.499199 19725 hybrid_clock.cc:648] HybridClock initialized: now 1790403772499192 us; error 34 us; skew 500 ppm
Sep 26 06:22:52 dist-test-slave-pd0t krb5kdc[18659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403772, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:22:52.501437 19725 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:22:52.501890 19725 webserver.cc:460] Webserver started at http://127.4.202.190:39647/ using document root <none> and password file <none>
I20260926 06:22:52.502106 19725 fs_manager.cc:359] Metadata directory not provided
I20260926 06:22:52.502173 19725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:22:52.503539 19725 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:22:52.504283 19740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:52.504469 19725 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:22:52.504581 19725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "43b852defc4e495187ed2c8cb041f43f"
format_stamp: "Formatted at 2026-09-26 06:22:50 on dist-test-slave-pd0t"
server_key: "e6277eec2f5b03889b4c1fa590e30de2"
server_key_iv: "e1463e74589144647f9100985fcf9009"
server_key_version: "encryptionkey@0"
I20260926 06:22:52.504896 19725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:22:52.520742 19725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:22:52.522667 19725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:22:52.522822 19725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:22:52.526505 19725 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:42879
I20260926 06:22:52.526572 19792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:42879 every 8 connection(s)
I20260926 06:22:52.526968 19725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:22:52.528430 19793 sys_catalog.cc:263] Verifying existing consensus state
I20260926 06:22:52.529066 19793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f: Bootstrap starting.
I20260926 06:22:52.534497 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 19725
I20260926 06:22:52.534853 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.129:46793
--local_ip_for_outbound_sockets=127.4.202.129
--tserver_master_addrs=127.4.202.190:42879
--webserver_port=46665
--webserver_interface=127.4.202.129
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:34237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:22:52.537876 19793 log.cc:824] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f: Log is configured to *not* fsync() on all Append() calls
I20260926 06:22:52.540189 19793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 06:22:52.540480 19793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f: Bootstrap complete.
I20260926 06:22:52.542658 19793 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b852defc4e495187ed2c8cb041f43f" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42879 } }
I20260926 06:22:52.542896 19793 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43b852defc4e495187ed2c8cb041f43f, State: Initialized, Role: FOLLOWER
I20260926 06:22:52.543061 19793 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b852defc4e495187ed2c8cb041f43f" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42879 } }
I20260926 06:22:52.543116 19793 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:22:52.543177 19793 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:22:52.543226 19793 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [term 2 FOLLOWER]: Advancing to term 3
I20260926 06:22:52.544189 19793 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b852defc4e495187ed2c8cb041f43f" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42879 } }
I20260926 06:22:52.544272 19793 leader_election.cc:302] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43b852defc4e495187ed2c8cb041f43f; no voters:
I20260926 06:22:52.544462 19793 leader_election.cc:288] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [CANDIDATE]: Term 3 election: Requested vote from peers
I20260926 06:22:52.544497 19797 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [term 3 FOLLOWER]: Leader election won for term 3
I20260926 06:22:52.544612 19797 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [term 3 LEADER]: Becoming Leader. State: Replica: 43b852defc4e495187ed2c8cb041f43f, State: Running, Role: LEADER
I20260926 06:22:52.544709 19793 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:22:52.544699 19797 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b852defc4e495187ed2c8cb041f43f" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42879 } }
I20260926 06:22:52.544956 19797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43b852defc4e495187ed2c8cb041f43f. Latest consensus state: current_term: 3 leader_uuid: "43b852defc4e495187ed2c8cb041f43f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b852defc4e495187ed2c8cb041f43f" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42879 } } }
I20260926 06:22:52.545046 19797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [sys.catalog]: This master's current role is: LEADER
I20260926 06:22:52.544955 19799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "43b852defc4e495187ed2c8cb041f43f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b852defc4e495187ed2c8cb041f43f" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42879 } } }
I20260926 06:22:52.545318 19799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43b852defc4e495187ed2c8cb041f43f [sys.catalog]: This master's current role is: LEADER
I20260926 06:22:52.545459 19801 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:22:52.546186 19801 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:22:52.546334 19801 catalog_manager.cc:1259] Loaded cluster ID: ce89315171d34522a610eed6b3b403d5
I20260926 06:22:52.546563 19801 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260926 06:22:52.546864 19801 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fef708489ad google::LogMessage::Fail() at ??:0
@ 0x7fef7084b88a google::LogMessage::SendToLog() at ??:0
@ 0x7fef708484de google::LogMessage::Flush() at ??:0
@ 0x7fef70849a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fef787690f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fef787728b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fef727160d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fef7270b954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fef7356f6db start_thread at ??:0
@ 0x7fef6f34661f clone at ??:0
W20260926 06:22:52.634711 19795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:22:52.634920 19795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:22:52.634955 19795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:22:52.636087 19795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:22:52.636153 19795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:22:52.636200 19795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.129
I20260926 06:22:52.637660 19795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:34237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.129:46793
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.202.129
--webserver_port=46665
--tserver_master_addrs=127.4.202.190:42879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.129
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:22:52.637933 19795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:22:52.638155 19795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:22:52.644050 19830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:52.644078 19832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:52.644212 19829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:22:52.644622 19795 server_base.cc:1034] running on GCE node
I20260926 06:22:52.644838 19795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:22:52.645126 19795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:22:52.646312 19795 hybrid_clock.cc:648] HybridClock initialized: now 1790403772646290 us; error 34 us; skew 500 ppm
Sep 26 06:22:52 dist-test-slave-pd0t krb5kdc[18659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403772, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:22:52.649540 19795 init.cc:377] Logged in from keytab as kudu/127.4.202.129@KRBTEST.COM (short username kudu)
I20260926 06:22:52.650077 19795 webserver.cc:460] Webserver started at http://127.4.202.129:46665/ using document root <none> and password file <none>
I20260926 06:22:52.650393 19795 fs_manager.cc:359] Metadata directory not provided
I20260926 06:22:52.650472 19795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:22:52.652177 19795 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:22:52.652928 19839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:52.653122 19795 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:22:52.653206 19795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/wal
uuid: "42e3a388aa6a4797afbbc09559b9658f"
format_stamp: "Formatted at 2026-09-26 06:22:51 on dist-test-slave-pd0t"
server_key: "9f573e3bc6b4a7b6d5691861c6996f7f"
server_key_iv: "90b3ac96a9e0d75ff77cf2d53635340a"
server_key_version: "encryptionkey@0"
I20260926 06:22:52.653548 19795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:22:52.675382 19795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:22:52.677704 19795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:22:52.677860 19795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:22:52.678102 19795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:22:52.678544 19846 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 06:22:52.680498 19795 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 06:22:52.680558 19795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260926 06:22:52.680600 19795 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 06:22:52.681211 19795 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 06:22:52.681267 19795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260926 06:22:52.681332 19846 tablet_bootstrap.cc:492] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f: Bootstrap starting.
I20260926 06:22:52.688392 19795 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.129:46793
I20260926 06:22:52.688529 19953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.129:46793 every 8 connection(s)
I20260926 06:22:52.688813 19795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
I20260926 06:22:52.692351 19846 log.cc:824] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f: Log is configured to *not* fsync() on all Append() calls
I20260926 06:22:52.694319 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 19795
I20260926 06:22:52.694628 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.130:43573
--local_ip_for_outbound_sockets=127.4.202.130
--tserver_master_addrs=127.4.202.190:42879
--webserver_port=41525
--webserver_interface=127.4.202.130
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:34237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:22:52.695245 19846 tablet_bootstrap.cc:492] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 06:22:52.695561 19846 tablet_bootstrap.cc:492] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f: Bootstrap complete.
I20260926 06:22:52.696835 19846 ts_tablet_manager.cc:1397] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.006s
I20260926 06:22:52.698269 19846 raft_consensus.cc:348] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf6d196a91e4256a94d85a8f82b4cc8" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43573 } } peers { permanent_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40851 } } peers { permanent_uuid: "42e3a388aa6a4797afbbc09559b9658f" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 46793 } }
I20260926 06:22:52.698662 19846 raft_consensus.cc:729] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42e3a388aa6a4797afbbc09559b9658f, State: Initialized, Role: FOLLOWER
I20260926 06:22:52.698829 19846 consensus_queue.cc:260] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf6d196a91e4256a94d85a8f82b4cc8" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43573 } } peers { permanent_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40851 } } peers { permanent_uuid: "42e3a388aa6a4797afbbc09559b9658f" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 46793 } }
I20260926 06:22:52.699096 19846 ts_tablet_manager.cc:1428] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
W20260926 06:22:52.749642 19956 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:22:52.689481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.129:50661)
0926 06:22:52.689699 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:52.689747 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:52.690187 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:52.749436 (+ 59249us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":99,"threads_started":1}
W20260926 06:22:52.749883 19954 heartbeater.cc:644] Failed to heartbeat to 127.4.202.190:42879 (0 consecutive failures): Network error: Failed to ping master at 127.4.202.190:42879: Client connection negotiation failed: client connection to 127.4.202.190:42879: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 06:22:52.750217 19956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:52.750088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.129:55819)
0926 06:22:52.750141 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:52.750181 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 06:22:52.750553 19956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:52.750430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.129:60403)
0926 06:22:52.750482 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:52.750524 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260926 06:22:52.750679 19954 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 06:22:52.775032 19958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:22:52.775207 19958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:22:52.775230 19958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:22:52.776296 19958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:22:52.776329 19958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:22:52.776355 19958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.130
I20260926 06:22:52.777688 19958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:34237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.130:43573
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.202.130
--webserver_port=41525
--tserver_master_addrs=127.4.202.190:42879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.130
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:22:52.777861 19958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:22:52.778018 19958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:22:52.782781 19964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:52.782787 19967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:22:52.782814 19958 server_base.cc:1034] running on GCE node
W20260926 06:22:52.782815 19965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:22:52.783192 19958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:22:52.783382 19958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:22:52.784519 19958 hybrid_clock.cc:648] HybridClock initialized: now 1790403772784508 us; error 27 us; skew 500 ppm
Sep 26 06:22:52 dist-test-slave-pd0t krb5kdc[18659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403772, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:22:52.786875 19958 init.cc:377] Logged in from keytab as kudu/127.4.202.130@KRBTEST.COM (short username kudu)
I20260926 06:22:52.787303 19958 webserver.cc:460] Webserver started at http://127.4.202.130:41525/ using document root <none> and password file <none>
I20260926 06:22:52.787537 19958 fs_manager.cc:359] Metadata directory not provided
I20260926 06:22:52.787604 19958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:22:52.788862 19958 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:22:52.789520 19974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:52.789700 19958 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:22:52.789788 19958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/wal
uuid: "edf6d196a91e4256a94d85a8f82b4cc8"
format_stamp: "Formatted at 2026-09-26 06:22:51 on dist-test-slave-pd0t"
server_key: "41d2b5ca8edfb75534d0651171d8be90"
server_key_iv: "d722cea4dc482c485608577559f94195"
server_key_version: "encryptionkey@0"
I20260926 06:22:52.790120 19958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:22:52.806365 19958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:22:52.808199 19958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:22:52.808358 19958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:22:52.808570 19958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:22:52.808851 19958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:22:52.808905 19958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:52.808943 19958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:22:52.808969 19958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:52.814868 19958 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.130:43573
I20260926 06:22:52.814985 20087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.130:43573 every 8 connection(s)
I20260926 06:22:52.815374 19958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
I20260926 06:22:52.816362 20090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:52.816022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.130:38901)
0926 06:22:52.816163 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:52.816225 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":68,"threads_started":1}
W20260926 06:22:52.816725 20088 heartbeater.cc:644] Failed to heartbeat to 127.4.202.190:42879 (0 consecutive failures): Network error: Failed to ping master at 127.4.202.190:42879: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
I20260926 06:22:52.816993 20090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:52.816907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.130:36187)
0926 06:22:52.816951 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:52.816977 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260926 06:22:52.817203 20090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:52.817155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.130:36233)
0926 06:22:52.817175 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:52.817192 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260926 06:22:52.817330 20088 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 06:22:52.819556 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 19958
I20260926 06:22:52.819768 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.131:40851
--local_ip_for_outbound_sockets=127.4.202.131
--tserver_master_addrs=127.4.202.190:42879
--webserver_port=44251
--webserver_interface=127.4.202.131
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:34237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:22:52.897898 20091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:22:52.898114 20091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:22:52.898151 20091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:22:52.899299 20091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:22:52.899363 20091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:22:52.899403 20091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.131
I20260926 06:22:52.900803 20091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:34237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.131:40851
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.202.131
--webserver_port=44251
--tserver_master_addrs=127.4.202.190:42879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.131
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:22:52.901048 20091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:22:52.901270 20091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:22:52.906206 20099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:22:52.906275 20091 server_base.cc:1034] running on GCE node
W20260926 06:22:52.906203 20096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:22:52.906199 20097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:22:52.906569 20091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:22:52.906925 20091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:22:52.908088 20091 hybrid_clock.cc:648] HybridClock initialized: now 1790403772908072 us; error 28 us; skew 500 ppm
Sep 26 06:22:52 dist-test-slave-pd0t krb5kdc[18659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403772, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:22:52.910512 20091 init.cc:377] Logged in from keytab as kudu/127.4.202.131@KRBTEST.COM (short username kudu)
I20260926 06:22:52.911015 20091 webserver.cc:460] Webserver started at http://127.4.202.131:44251/ using document root <none> and password file <none>
I20260926 06:22:52.911248 20091 fs_manager.cc:359] Metadata directory not provided
I20260926 06:22:52.911316 20091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:22:52.912573 20091 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:22:52.913151 20106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:52.913316 20091 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:52.913410 20091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/wal
uuid: "5e2dd6c2c8e245038da7f222b8dd7850"
format_stamp: "Formatted at 2026-09-26 06:22:51 on dist-test-slave-pd0t"
server_key: "6e6ba8d54dc704ef4de3bb13e5750299"
server_key_iv: "df546a04578b5850d57da5890577398e"
server_key_version: "encryptionkey@0"
I20260926 06:22:52.913724 20091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:22:52.928149 20091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:22:52.929848 20091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:22:52.929999 20091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:22:52.930202 20091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:22:52.930502 20091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:22:52.930552 20091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:52.930601 20091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:22:52.930624 20091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:22:52.936633 20091 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.131:40851
I20260926 06:22:52.936676 20219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.131:40851 every 8 connection(s)
I20260926 06:22:52.936998 20091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
I20260926 06:22:52.938077 20222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:52.937691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.131:38409)
0926 06:22:52.937856 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:52.937927 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":67,"threads_started":1}
W20260926 06:22:52.938529 20220 heartbeater.cc:644] Failed to heartbeat to 127.4.202.190:42879 (0 consecutive failures): Network error: Failed to ping master at 127.4.202.190:42879: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
I20260926 06:22:52.938849 20222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:52.938745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.131:52501)
0926 06:22:52.938789 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:52.938830 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260926 06:22:52.939167 20222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:52.939093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.131:48495)
0926 06:22:52.939131 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:52.939152 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260926 06:22:52.939342 20220 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 06:22:52.944015 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 20091
I20260926 06:22:53.751149 20224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:53.750947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.129:58215)
0926 06:22:53.751065 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:53.751092 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":68,"threads_started":1}
I20260926 06:22:53.817809 20225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:53.817615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.130:56197)
0926 06:22:53.817727 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:53.817757 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":53,"threads_started":1}
I20260926 06:22:53.857003 20226 raft_consensus.cc:482] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:22:53.857149 20226 raft_consensus.cc:504] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf6d196a91e4256a94d85a8f82b4cc8" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43573 } } peers { permanent_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40851 } } peers { permanent_uuid: "42e3a388aa6a4797afbbc09559b9658f" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 46793 } }
I20260926 06:22:53.857606 20226 leader_election.cc:288] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers edf6d196a91e4256a94d85a8f82b4cc8 (127.4.202.130:43573), 5e2dd6c2c8e245038da7f222b8dd7850 (127.4.202.131:40851)
Sep 26 06:22:53 dist-test-slave-pd0t krb5kdc[18659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403772, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for kudu/127.4.202.130@KRBTEST.COM
Sep 26 06:22:53 dist-test-slave-pd0t krb5kdc[18659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403772, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for kudu/127.4.202.131@KRBTEST.COM
I20260926 06:22:53.868534 20228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:53.857840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:41099 (local address 127.4.202.130:43573)
0926 06:22:53.858089 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:53.858097 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:53.858122 (+ 25us) server_negotiation.cc:407] Connection header received
0926 06:22:53.858522 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:53.858524 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:53.858559 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:53.858775 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:53.859557 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:53.860455 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:53.861876 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:53.862254 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:53.862393 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:53.864728 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:53.864733 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:53.864733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:53.864745 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:53.866802 (+ 2057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:53.867251 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:53.867254 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:53.867254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:53.867369 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:53.867654 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:53.867656 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:53.867656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:53.867922 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:53.867984 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:53.868302 (+ 318us) server_negotiation.cc:299] Negotiation successful
0926 06:22:53.868427 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":61,"threads_started":1}
I20260926 06:22:53.868525 20227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:53.857746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:40851 (local address 127.4.202.129:42087)
0926 06:22:53.857894 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:53.857903 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:53.857946 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:53.858795 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:53.858798 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:53.859198 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:22:53.859395 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:53.859400 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:53.860670 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:53.860671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:53.861792 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:53.861795 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:53.862419 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:53.862420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:53.862498 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:53.863176 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:22:53.863181 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:22:53.865057 (+ 1876us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:22:53.867056 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:53.867056 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:53.867057 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:53.867268 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:53.867588 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:53.867588 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:53.867589 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:53.867666 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:53.868044 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:22:53.868045 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:22:53.868130 (+ 85us) client_negotiation.cc:769] Sending connection context
0926 06:22:53.868234 (+ 104us) client_negotiation.cc:240] Negotiation successful
0926 06:22:53.868424 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":37,"thread_start_us":44,"threads_started":1}
I20260926 06:22:53.868582 20229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:53.857920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:42087 (local address 127.4.202.131:40851)
0926 06:22:53.858032 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0926 06:22:53.858043 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 06:22:53.858066 (+ 23us) server_negotiation.cc:407] Connection header received
0926 06:22:53.858483 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:22:53.858485 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:22:53.858536 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:22:53.858770 (+ 234us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:22:53.859520 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:53.860578 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:53.861917 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:22:53.862325 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:22:53.862442 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:53.865183 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:22:53.865186 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:22:53.865186 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:22:53.865196 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:22:53.866974 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:53.867375 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:53.867377 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:53.867378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:53.867518 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:22:53.867732 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:22:53.867733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:22:53.867733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:22:53.867945 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:22:53.868048 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 06:22:53.868325 (+ 277us) server_negotiation.cc:299] Negotiation successful
0926 06:22:53.868464 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74,"thread_start_us":49,"threads_started":1}
I20260926 06:22:53.868630 20224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:53.857730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:43573 (local address 127.4.202.129:41099)
0926 06:22:53.857848 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:53.857858 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:22:53.857931 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:22:53.858795 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:22:53.858798 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:22:53.859215 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:22:53.859407 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:53.859410 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:53.860553 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:53.860554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:53.861792 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:22:53.861795 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:22:53.862362 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:22:53.862364 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:22:53.862485 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:22:53.863145 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:22:53.863154 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:22:53.864612 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:22:53.866878 (+ 2266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:53.866885 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:53.866887 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:53.867172 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:53.867454 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:22:53.867454 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:22:53.867455 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:22:53.867587 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:22:53.867996 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:22:53.868002 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:22:53.868140 (+ 138us) client_negotiation.cc:769] Sending connection context
0926 06:22:53.868233 (+ 93us) client_negotiation.cc:240] Negotiation successful
0926 06:22:53.868499 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":213}
W20260926 06:22:53.869686 19843 leader_election.cc:341] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer edf6d196a91e4256a94d85a8f82b4cc8 (127.4.202.130:43573): Not found: Tablet not found: 6f3f8b7fcdfb406c952d54ac848b382c
W20260926 06:22:53.869768 19842 leader_election.cc:341] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e2dd6c2c8e245038da7f222b8dd7850 (127.4.202.131:40851): Not found: Tablet not found: 6f3f8b7fcdfb406c952d54ac848b382c
I20260926 06:22:53.869848 19842 leader_election.cc:302] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 42e3a388aa6a4797afbbc09559b9658f; no voters: 5e2dd6c2c8e245038da7f222b8dd7850, edf6d196a91e4256a94d85a8f82b4cc8
I20260926 06:22:53.869938 20226 raft_consensus.cc:2726] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 06:22:53.939810 20230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:53.939609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.131:50359)
0926 06:22:53.939742 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:53.939777 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":63,"threads_started":1}
I20260926 06:22:54.751802 20231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:54.751601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.129:40015)
0926 06:22:54.751719 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:54.751761 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":59,"threads_started":1}
I20260926 06:22:54.818389 20232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:54.818202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.130:60167)
0926 06:22:54.818311 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:54.818355 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":56,"threads_started":1}
I20260926 06:22:54.940402 20233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:54.940210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.131:53665)
0926 06:22:54.940324 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:54.940355 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":48,"threads_started":1}
I20260926 06:22:55.668431 20234 raft_consensus.cc:482] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:22:55.668545 20234 raft_consensus.cc:504] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf6d196a91e4256a94d85a8f82b4cc8" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43573 } } peers { permanent_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40851 } } peers { permanent_uuid: "42e3a388aa6a4797afbbc09559b9658f" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 46793 } }
I20260926 06:22:55.668774 20234 leader_election.cc:288] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers edf6d196a91e4256a94d85a8f82b4cc8 (127.4.202.130:43573), 5e2dd6c2c8e245038da7f222b8dd7850 (127.4.202.131:40851)
W20260926 06:22:55.669190 19843 leader_election.cc:341] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer edf6d196a91e4256a94d85a8f82b4cc8 (127.4.202.130:43573): Not found: Tablet not found: 6f3f8b7fcdfb406c952d54ac848b382c
W20260926 06:22:55.669296 19842 leader_election.cc:341] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e2dd6c2c8e245038da7f222b8dd7850 (127.4.202.131:40851): Not found: Tablet not found: 6f3f8b7fcdfb406c952d54ac848b382c
I20260926 06:22:55.669339 19842 leader_election.cc:302] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 42e3a388aa6a4797afbbc09559b9658f; no voters: 5e2dd6c2c8e245038da7f222b8dd7850, edf6d196a91e4256a94d85a8f82b4cc8
I20260926 06:22:55.669436 20234 raft_consensus.cc:2726] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 06:22:55.752447 20235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:55.752234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.129:34411)
0926 06:22:55.752357 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:55.752391 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":81,"threads_started":1}
I20260926 06:22:55.819046 20236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:55.818827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.130:57777)
0926 06:22:55.818947 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:55.818985 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":64,"threads_started":1}
I20260926 06:22:55.940985 20237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:55.940795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.131:35499)
0926 06:22:55.940920 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:55.940949 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":55,"threads_started":1}
I20260926 06:22:56.753072 20238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:56.752870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.129:39751)
0926 06:22:56.752992 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:56.753026 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":57,"threads_started":1}
I20260926 06:22:56.819665 20239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:56.819478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.130:45023)
0926 06:22:56.819600 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:56.819631 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":66,"threads_started":1}
I20260926 06:22:56.941695 20240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:56.941473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.131:37291)
0926 06:22:56.941624 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:56.941657 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":80,"threads_started":1}
I20260926 06:22:57.753788 20241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:57.753600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.129:48973)
0926 06:22:57.753711 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:57.753739 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":55,"threads_started":1}
I20260926 06:22:57.820252 20242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:57.820071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.130:59413)
0926 06:22:57.820185 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:57.820217 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":57,"threads_started":1}
I20260926 06:22:57.942312 20243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:57.942117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.131:43483)
0926 06:22:57.942239 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:57.942271 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":69,"threads_started":1}
I20260926 06:22:58.615383 20244 raft_consensus.cc:482] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:22:58.615471 20244 raft_consensus.cc:504] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf6d196a91e4256a94d85a8f82b4cc8" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43573 } } peers { permanent_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40851 } } peers { permanent_uuid: "42e3a388aa6a4797afbbc09559b9658f" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 46793 } }
I20260926 06:22:58.615625 20244 leader_election.cc:288] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers edf6d196a91e4256a94d85a8f82b4cc8 (127.4.202.130:43573), 5e2dd6c2c8e245038da7f222b8dd7850 (127.4.202.131:40851)
W20260926 06:22:58.615938 19843 leader_election.cc:341] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer edf6d196a91e4256a94d85a8f82b4cc8 (127.4.202.130:43573): Not found: Tablet not found: 6f3f8b7fcdfb406c952d54ac848b382c
W20260926 06:22:58.616035 19842 leader_election.cc:341] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e2dd6c2c8e245038da7f222b8dd7850 (127.4.202.131:40851): Not found: Tablet not found: 6f3f8b7fcdfb406c952d54ac848b382c
I20260926 06:22:58.616120 19842 leader_election.cc:302] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 42e3a388aa6a4797afbbc09559b9658f; no voters: 5e2dd6c2c8e245038da7f222b8dd7850, edf6d196a91e4256a94d85a8f82b4cc8
I20260926 06:22:58.616211 20244 raft_consensus.cc:2726] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 06:22:58.754462 20245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:58.754265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.129:53561)
0926 06:22:58.754386 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:58.754417 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":63,"threads_started":1}
I20260926 06:22:58.820874 20246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:58.820690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.130:41475)
0926 06:22:58.820803 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:58.820836 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
I20260926 06:22:58.942958 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:58.942752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.131:60349)
0926 06:22:58.942886 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:58.942920 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":72,"threads_started":1}
I20260926 06:22:59.755172 20248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:59.754939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.129:47383)
0926 06:22:59.755060 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:59.755098 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":62,"threads_started":1}
I20260926 06:22:59.821481 20249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:59.821306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.130:51429)
0926 06:22:59.821413 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:59.821445 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":55,"threads_started":1}
I20260926 06:22:59.943612 20250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:22:59.943407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.131:46285)
0926 06:22:59.943542 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 06:22:59.943574 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":74,"threads_started":1}
I20260926 06:23:00.755810 20251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:00.755603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.129:53511)
0926 06:23:00.755733 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:00.755766 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":59,"threads_started":1}
I20260926 06:23:00.822196 20252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:00.821977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.130:50849)
0926 06:23:00.822090 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:00.822147 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":68,"threads_started":1}
I20260926 06:23:00.944362 20253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:00.944137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.131:47567)
0926 06:23:00.944285 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:00.944319 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":63,"threads_started":1}
I20260926 06:23:01.756486 20254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:01.756291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.129:33691)
0926 06:23:01.756406 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:01.756442 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":60,"threads_started":1}
I20260926 06:23:01.822870 20255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:01.822683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.130:46695)
0926 06:23:01.822791 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:01.822828 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":55,"threads_started":1}
I20260926 06:23:01.945024 20256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:01.944827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.131:50285)
0926 06:23:01.944954 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:01.944986 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":69,"threads_started":1}
I20260926 06:23:02.757171 20257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:02.756956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.129:59277)
0926 06:23:02.757084 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:02.757126 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":57,"threads_started":1}
I20260926 06:23:02.823451 20258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:02.823279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.130:34091)
0926 06:23:02.823385 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:02.823414 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":55,"threads_started":1}
I20260926 06:23:02.945676 20259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:02.945457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.131:55077)
0926 06:23:02.945610 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:02.945639 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":74,"threads_started":1}
I20260926 06:23:03.517542 20260 raft_consensus.cc:482] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:23:03.517627 20260 raft_consensus.cc:504] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf6d196a91e4256a94d85a8f82b4cc8" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43573 } } peers { permanent_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40851 } } peers { permanent_uuid: "42e3a388aa6a4797afbbc09559b9658f" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 46793 } }
I20260926 06:23:03.517838 20260 leader_election.cc:288] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers edf6d196a91e4256a94d85a8f82b4cc8 (127.4.202.130:43573), 5e2dd6c2c8e245038da7f222b8dd7850 (127.4.202.131:40851)
W20260926 06:23:03.518275 19842 leader_election.cc:341] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e2dd6c2c8e245038da7f222b8dd7850 (127.4.202.131:40851): Not found: Tablet not found: 6f3f8b7fcdfb406c952d54ac848b382c
W20260926 06:23:03.518340 19843 leader_election.cc:341] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer edf6d196a91e4256a94d85a8f82b4cc8 (127.4.202.130:43573): Not found: Tablet not found: 6f3f8b7fcdfb406c952d54ac848b382c
I20260926 06:23:03.518366 19842 leader_election.cc:302] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 42e3a388aa6a4797afbbc09559b9658f; no voters: 5e2dd6c2c8e245038da7f222b8dd7850, edf6d196a91e4256a94d85a8f82b4cc8
I20260926 06:23:03.518497 20260 raft_consensus.cc:2726] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 06:23:03.757754 20261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:03.757561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.129:37931)
0926 06:23:03.757676 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:03.757709 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":58,"threads_started":1}
I20260926 06:23:03.824084 20262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:03.823880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.130:38029)
0926 06:23:03.824004 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:03.824036 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":69,"threads_started":1}
I20260926 06:23:03.946343 20263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:03.946130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.131:53947)
0926 06:23:03.946266 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:03.946301 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":59,"threads_started":1}
I20260926 06:23:04.758394 20264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:04.758168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.129:36245)
0926 06:23:04.758314 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:04.758348 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":61,"threads_started":1}
I20260926 06:23:04.824730 20265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:04.824494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.130:33811)
0926 06:23:04.824647 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:04.824688 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":77,"threads_started":1}
I20260926 06:23:04.947099 20266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:04.946895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.131:38709)
0926 06:23:04.947022 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:04.947056 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
I20260926 06:23:05.759043 20267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:05.758791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.129:34911)
0926 06:23:05.758936 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:05.758970 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":56,"threads_started":1}
I20260926 06:23:05.825446 20268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:05.825275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.130:59169)
0926 06:23:05.825380 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:05.825410 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":54,"threads_started":1}
I20260926 06:23:05.947757 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:05.947482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.131:34633)
0926 06:23:05.947649 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:05.947716 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":77,"threads_started":1}
I20260926 06:23:06.163602 20270 raft_consensus.cc:482] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:23:06.163687 20270 raft_consensus.cc:504] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf6d196a91e4256a94d85a8f82b4cc8" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43573 } } peers { permanent_uuid: "5e2dd6c2c8e245038da7f222b8dd7850" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40851 } } peers { permanent_uuid: "42e3a388aa6a4797afbbc09559b9658f" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 46793 } }
I20260926 06:23:06.163849 20270 leader_election.cc:288] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers edf6d196a91e4256a94d85a8f82b4cc8 (127.4.202.130:43573), 5e2dd6c2c8e245038da7f222b8dd7850 (127.4.202.131:40851)
W20260926 06:23:06.164238 19842 leader_election.cc:341] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e2dd6c2c8e245038da7f222b8dd7850 (127.4.202.131:40851): Not found: Tablet not found: 6f3f8b7fcdfb406c952d54ac848b382c
W20260926 06:23:06.164321 19843 leader_election.cc:341] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer edf6d196a91e4256a94d85a8f82b4cc8 (127.4.202.130:43573): Not found: Tablet not found: 6f3f8b7fcdfb406c952d54ac848b382c
I20260926 06:23:06.164393 19843 leader_election.cc:302] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 42e3a388aa6a4797afbbc09559b9658f; no voters: 5e2dd6c2c8e245038da7f222b8dd7850, edf6d196a91e4256a94d85a8f82b4cc8
I20260926 06:23:06.164489 20270 raft_consensus.cc:2726] T 6f3f8b7fcdfb406c952d54ac848b382c P 42e3a388aa6a4797afbbc09559b9658f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 06:23:06.759851 20271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:06.759515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.129:54383)
0926 06:23:06.759772 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:06.759807 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":71,"threads_started":1}
I20260926 06:23:06.826051 20272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:06.825851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.130:37955)
0926 06:23:06.825960 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:06.825993 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":55,"threads_started":1}
I20260926 06:23:06.948361 20273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:06.948153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.131:37347)
0926 06:23:06.948285 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:06.948319 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":57,"threads_started":1}
I20260926 06:23:07.760515 20274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:07.760294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.129:48273)
0926 06:23:07.760442 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:07.760473 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":58,"threads_started":1}
I20260926 06:23:07.826843 20275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:07.826648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.130:38317)
0926 06:23:07.826770 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:07.826802 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":60,"threads_started":1}
I20260926 06:23:07.944793 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 19795
I20260926 06:23:07.949102 20276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:07.948915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42879 (local address 127.4.202.131:34861)
0926 06:23:07.949031 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:07.949067 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":70,"threads_started":1}
I20260926 06:23:07.949671 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 19958
I20260926 06:23:07.954071 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 20091
2026-09-26T06:23:07Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18240 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:23:08 dist-test-slave-pd0t krb5kdc[20279](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:23:08 dist-test-slave-pd0t krb5kdc[20279](info): set up 2 sockets
Sep 26 06:23:08 dist-test-slave-pd0t krb5kdc[20279](info): commencing operation
krb5kdc: starting...
W20260926 06:23:08.982787 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:23:09 dist-test-slave-pd0t krb5kdc[20279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:23:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:23:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.202.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
I20260926 06:23:09.091581 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:41615
--webserver_interface=127.4.202.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:41921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:41615
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:23:09.172137 20292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:23:09.172344 20292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:23:09.172370 20292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:23:09.173497 20292 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 06:23:09.173537 20292 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 06:23:09.173576 20292 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 06:23:09.173599 20292 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:23:09.173632 20292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:23:09.173655 20292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:23:09.173671 20292 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:23:09.173684 20292 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:23:09.175149 20292 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:41921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:41615
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:41615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:23:09.175329 20292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:23:09.175483 20292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:23:09.180575 20298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:23:09.180622 20300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:23:09.180665 20292 server_base.cc:1034] running on GCE node
W20260926 06:23:09.180622 20297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:23:09.181042 20292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:23:09.181334 20292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:23:09.182484 20292 hybrid_clock.cc:648] HybridClock initialized: now 1790403789182477 us; error 30 us; skew 500 ppm
Sep 26 06:23:09 dist-test-slave-pd0t krb5kdc[20279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403789, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:23:09.184928 20292 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:23:09.185396 20292 webserver.cc:460] Webserver started at http://127.4.202.190:46051/ using document root <none> and password file <none>
I20260926 06:23:09.185626 20292 fs_manager.cc:359] Metadata directory not provided
I20260926 06:23:09.185668 20292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:23:09.185842 20292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:23:09.186890 20292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "90b703134eac456597feb5ca43537099"
format_stamp: "Formatted at 2026-09-26 06:23:09 on dist-test-slave-pd0t"
server_key: "931169c4c3d8c12ba5b4407b7abb25a4"
server_key_iv: "3d8f79484670ed072ea8083ad0646833"
server_key_version: "encryptionkey@0"
I20260926 06:23:09.187227 20292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "90b703134eac456597feb5ca43537099"
format_stamp: "Formatted at 2026-09-26 06:23:09 on dist-test-slave-pd0t"
server_key: "931169c4c3d8c12ba5b4407b7abb25a4"
server_key_iv: "3d8f79484670ed072ea8083ad0646833"
server_key_version: "encryptionkey@0"
I20260926 06:23:09.188442 20292 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:23:09.189121 20307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:23:09.189316 20292 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:23:09.189400 20292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "90b703134eac456597feb5ca43537099"
format_stamp: "Formatted at 2026-09-26 06:23:09 on dist-test-slave-pd0t"
server_key: "931169c4c3d8c12ba5b4407b7abb25a4"
server_key_iv: "3d8f79484670ed072ea8083ad0646833"
server_key_version: "encryptionkey@0"
I20260926 06:23:09.189478 20292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:23:09.198534 20292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:23:09.200251 20292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:23:09.200397 20292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:23:09.204267 20292 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:41615
I20260926 06:23:09.204303 20359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:41615 every 8 connection(s)
I20260926 06:23:09.204617 20292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:23:09.205601 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 20292
I20260926 06:23:09.205646 20360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:23:09.205749 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:23:09.206007 4906 external_mini_cluster.cc:1428] Setting key b93b43eee9f2eb018f9e6a5150910f8e
I20260926 06:23:09.208827 20360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099: Bootstrap starting.
I20260926 06:23:09.209550 20360 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099: Neither blocks nor log segments found. Creating new log.
I20260926 06:23:09.209841 20360 log.cc:824] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099: Log is configured to *not* fsync() on all Append() calls
I20260926 06:23:09.210528 20360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099: No bootstrap required, opened a new log
Sep 26 06:23:09 dist-test-slave-pd0t krb5kdc[20279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:23:09.213254 20360 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90b703134eac456597feb5ca43537099" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 41615 } }
I20260926 06:23:09.213430 20360 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:23:09.213469 20360 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90b703134eac456597feb5ca43537099, State: Initialized, Role: FOLLOWER
I20260926 06:23:09.213605 20360 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90b703134eac456597feb5ca43537099" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 41615 } }
I20260926 06:23:09.213699 20360 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:23:09.213757 20360 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:23:09.213807 20360 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:23:09.214308 20360 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90b703134eac456597feb5ca43537099" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 41615 } }
I20260926 06:23:09.214427 20360 leader_election.cc:302] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90b703134eac456597feb5ca43537099; no voters:
I20260926 06:23:09.214633 20360 leader_election.cc:288] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:23:09.214668 20365 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:23:09.214836 20365 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [term 1 LEADER]: Becoming Leader. State: Replica: 90b703134eac456597feb5ca43537099, State: Running, Role: LEADER
I20260926 06:23:09.214879 20360 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:23:09.214985 20365 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90b703134eac456597feb5ca43537099" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 41615 } }
I20260926 06:23:09.215824 20367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90b703134eac456597feb5ca43537099. Latest consensus state: current_term: 1 leader_uuid: "90b703134eac456597feb5ca43537099" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90b703134eac456597feb5ca43537099" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 41615 } } }
I20260926 06:23:09.215919 20367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [sys.catalog]: This master's current role is: LEADER
I20260926 06:23:09.216128 20377 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:23:09.216248 20366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90b703134eac456597feb5ca43537099" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90b703134eac456597feb5ca43537099" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 41615 } } }
I20260926 06:23:09.216355 20366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [sys.catalog]: This master's current role is: LEADER
I20260926 06:23:09.217442 20363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:09.206513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39148 (local address 127.4.202.190:41615)
0926 06:23:09.206684 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 06:23:09.206692 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 06:23:09.206721 (+ 29us) server_negotiation.cc:407] Connection header received
0926 06:23:09.207183 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:23:09.207188 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:23:09.207259 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:23:09.207363 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:23:09.208139 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:09.209036 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:09.210112 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:09.210629 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:09.210735 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:09.213304 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:23:09.213307 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:23:09.213307 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:23:09.213318 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:23:09.215433 (+ 2115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:23:09.216108 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:23:09.216111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:23:09.216111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:23:09.216207 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:23:09.216403 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:23:09.216404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:23:09.216404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:23:09.216688 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:23:09.216871 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0926 06:23:09.217181 (+ 310us) server_negotiation.cc:299] Negotiation successful
0926 06:23:09.217307 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":142,"threads_started":1}
I20260926 06:23:09.217598 20377 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:23:09.219043 20377 catalog_manager.cc:1347] Generated new cluster ID: 286568cd652242be8b3d784b16bff7a8
I20260926 06:23:09.219092 20377 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:23:09.225324 20377 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:23:09.225777 20377 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:23:09.234251 20377 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099: Generated new TSK 0
I20260926 06:23:09.234444 20377 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.202.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
I20260926 06:23:09.265313 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.129:0
--local_ip_for_outbound_sockets=127.4.202.129
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:41615
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:41921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:23:09.344051 20386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:23:09.344280 20386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:23:09.344318 20386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:23:09.345424 20386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:23:09.345491 20386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:23:09.345535 20386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.129
I20260926 06:23:09.346980 20386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:41921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:41615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.129
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:23:09.347237 20386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:23:09.347460 20386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:23:09.352288 20392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:23:09.352370 20394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:23:09.352355 20386 server_base.cc:1034] running on GCE node
W20260926 06:23:09.352337 20391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:23:09.352695 20386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:23:09.352931 20386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:23:09.354068 20386 hybrid_clock.cc:648] HybridClock initialized: now 1790403789354049 us; error 37 us; skew 500 ppm
Sep 26 06:23:09 dist-test-slave-pd0t krb5kdc[20279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403789, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:23:09.356472 20386 init.cc:377] Logged in from keytab as kudu/127.4.202.129@KRBTEST.COM (short username kudu)
I20260926 06:23:09.356889 20386 webserver.cc:460] Webserver started at http://127.4.202.129:35235/ using document root <none> and password file <none>
I20260926 06:23:09.357054 20386 fs_manager.cc:359] Metadata directory not provided
I20260926 06:23:09.357092 20386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:23:09.357187 20386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:23:09.358137 20386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/data/instance:
uuid: "c8900ea32c9e4ca586522252586ec729"
format_stamp: "Formatted at 2026-09-26 06:23:09 on dist-test-slave-pd0t"
server_key: "25409706138cc6fc0dddab57397c20fa"
server_key_iv: "b5ea3da294d48df3e8520d03197b02ce"
server_key_version: "encryptionkey@0"
I20260926 06:23:09.358423 20386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance:
uuid: "c8900ea32c9e4ca586522252586ec729"
format_stamp: "Formatted at 2026-09-26 06:23:09 on dist-test-slave-pd0t"
server_key: "25409706138cc6fc0dddab57397c20fa"
server_key_iv: "b5ea3da294d48df3e8520d03197b02ce"
server_key_version: "encryptionkey@0"
I20260926 06:23:09.359694 20386 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260926 06:23:09.360340 20401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:23:09.360522 20386 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:23:09.360589 20386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/wal
uuid: "c8900ea32c9e4ca586522252586ec729"
format_stamp: "Formatted at 2026-09-26 06:23:09 on dist-test-slave-pd0t"
server_key: "25409706138cc6fc0dddab57397c20fa"
server_key_iv: "b5ea3da294d48df3e8520d03197b02ce"
server_key_version: "encryptionkey@0"
I20260926 06:23:09.360673 20386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:23:09.384994 20386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:23:09.386742 20386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:23:09.386880 20386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:23:09.387097 20386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:23:09.387394 20386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:23:09.387447 20386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:23:09.387480 20386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:23:09.387507 20386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:23:09.393251 20386 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.129:45397
I20260926 06:23:09.393304 20514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.129:45397 every 8 connection(s)
I20260926 06:23:09.393568 20386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
Sep 26 06:23:09 dist-test-slave-pd0t krb5kdc[20279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403789, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:23:09.400079 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 20386
I20260926 06:23:09.400202 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance
I20260926 06:23:09.400424 4906 external_mini_cluster.cc:1428] Setting key 0f6abd2c39a6ecd627f7817d13560ad0
I20260926 06:23:09.402279 20517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:09.394190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.129:59433)
0926 06:23:09.394331 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:09.394362 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0926 06:23:09.394809 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:23:09.394997 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:23:09.395000 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:23:09.395520 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:23:09.395695 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:09.395699 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:09.396337 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:09.396337 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:09.397111 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:09.397113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:09.397433 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:09.397433 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:09.397524 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:09.397940 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:23:09.397947 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:23:09.399292 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:23:09.401028 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:23:09.401034 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:23:09.401036 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:23:09.401294 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:23:09.401494 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:23:09.401494 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:23:09.401495 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:23:09.401586 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:23:09.401851 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:23:09.401855 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:23:09.401954 (+ 99us) client_negotiation.cc:769] Sending connection context
0926 06:23:09.402030 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 06:23:09.402136 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":84,"threads_started":1}
I20260926 06:23:09.402311 20363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:09.394190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:59433 (local address 127.4.202.190:41615)
0926 06:23:09.394238 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 06:23:09.394240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:23:09.394448 (+ 208us) server_negotiation.cc:407] Connection header received
0926 06:23:09.394891 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:23:09.394892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:23:09.394901 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:23:09.394953 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:23:09.395778 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:09.396260 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:09.397182 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:09.397364 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:09.397411 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:09.399384 (+ 1973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:23:09.399385 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:23:09.399385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:23:09.399391 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:23:09.400949 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:23:09.401369 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:23:09.401370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:23:09.401370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:23:09.401427 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:23:09.401647 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:23:09.401648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:23:09.401648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:23:09.401789 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:23:09.401854 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 06:23:09.402073 (+ 219us) server_negotiation.cc:299] Negotiation successful
0926 06:23:09.402162 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:23:09.403023 20515 heartbeater.cc:347] Connected to a master server at 127.4.202.190:41615
I20260926 06:23:09.403100 20515 heartbeater.cc:464] Registering TS with master...
I20260926 06:23:09.403288 20515 heartbeater.cc:511] Master 127.4.202.190:41615 requested a full tablet report, sending...
I20260926 06:23:09.403769 20324 ts_manager.cc:194] Registered new tserver with Master: c8900ea32c9e4ca586522252586ec729 (127.4.202.129:45397)
I20260926 06:23:09.404378 20324 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.129@KRBTEST.COM'} at 127.4.202.129:59433
WARNING: no policy specified for kudu/127.4.202.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
I20260926 06:23:09.425594 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.130:0
--local_ip_for_outbound_sockets=127.4.202.130
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:41615
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:41921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:23:09.503871 20520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:23:09.504055 20520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:23:09.504076 20520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:23:09.505192 20520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:23:09.505234 20520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:23:09.505259 20520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.130
I20260926 06:23:09.506752 20520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:41921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:41615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.130
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:23:09.506975 20520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:23:09.507186 20520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:23:09.511962 20528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:23:09.511982 20526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:23:09.511999 20525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:23:09.512040 20520 server_base.cc:1034] running on GCE node
I20260926 06:23:09.512369 20520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:23:09.512610 20520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:23:09.513763 20520 hybrid_clock.cc:648] HybridClock initialized: now 1790403789513750 us; error 38 us; skew 500 ppm
Sep 26 06:23:09 dist-test-slave-pd0t krb5kdc[20279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403789, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:23:09.516434 20520 init.cc:377] Logged in from keytab as kudu/127.4.202.130@KRBTEST.COM (short username kudu)
I20260926 06:23:09.516839 20520 webserver.cc:460] Webserver started at http://127.4.202.130:41613/ using document root <none> and password file <none>
I20260926 06:23:09.517009 20520 fs_manager.cc:359] Metadata directory not provided
I20260926 06:23:09.517047 20520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:23:09.517130 20520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:23:09.518062 20520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/data/instance:
uuid: "515692ec901c470b8b3d3f894cd8bb83"
format_stamp: "Formatted at 2026-09-26 06:23:09 on dist-test-slave-pd0t"
server_key: "30cc733beab913f1436d6311c50c8000"
server_key_iv: "8c8a47d736497e90060cd9acd6d59f6c"
server_key_version: "encryptionkey@0"
I20260926 06:23:09.518347 20520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance:
uuid: "515692ec901c470b8b3d3f894cd8bb83"
format_stamp: "Formatted at 2026-09-26 06:23:09 on dist-test-slave-pd0t"
server_key: "30cc733beab913f1436d6311c50c8000"
server_key_iv: "8c8a47d736497e90060cd9acd6d59f6c"
server_key_version: "encryptionkey@0"
I20260926 06:23:09.519654 20520 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:23:09.520300 20535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:23:09.520493 20520 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:23:09.520561 20520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/wal
uuid: "515692ec901c470b8b3d3f894cd8bb83"
format_stamp: "Formatted at 2026-09-26 06:23:09 on dist-test-slave-pd0t"
server_key: "30cc733beab913f1436d6311c50c8000"
server_key_iv: "8c8a47d736497e90060cd9acd6d59f6c"
server_key_version: "encryptionkey@0"
I20260926 06:23:09.520640 20520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:23:09.535239 20520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:23:09.537048 20520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:23:09.537163 20520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:23:09.537379 20520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:23:09.537597 20520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:23:09.537627 20520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:23:09.537647 20520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:23:09.537660 20520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:23:09.543318 20520 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.130:38333
I20260926 06:23:09.543401 20648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.130:38333 every 8 connection(s)
I20260926 06:23:09.543741 20520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
Sep 26 06:23:09 dist-test-slave-pd0t krb5kdc[20279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403789, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:23:09.549683 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 20520
I20260926 06:23:09.549808 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance
I20260926 06:23:09.549990 4906 external_mini_cluster.cc:1428] Setting key 1ae65911c09339db6947493bef26aa2a
I20260926 06:23:09.553181 20363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:09.544398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:52847 (local address 127.4.202.190:41615)
0926 06:23:09.544437 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 06:23:09.544440 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:23:09.544611 (+ 171us) server_negotiation.cc:407] Connection header received
0926 06:23:09.545140 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:23:09.545142 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:23:09.545155 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:23:09.545250 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:23:09.546161 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:09.546779 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:09.547743 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:09.547954 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:09.548038 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:09.550256 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:23:09.550258 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:23:09.550258 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:23:09.550265 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:23:09.551749 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:23:09.552205 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:23:09.552206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:23:09.552206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:23:09.552274 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:23:09.552540 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:23:09.552541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:23:09.552542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:23:09.552653 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:23:09.552730 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 06:23:09.553010 (+ 280us) server_negotiation.cc:299] Negotiation successful
0926 06:23:09.553089 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 06:23:09.553395 20651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:09.544361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.130:52847)
0926 06:23:09.544493 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:09.544535 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 06:23:09.545021 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:23:09.545320 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:23:09.545324 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:23:09.545879 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:23:09.546053 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:09.546057 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:09.546865 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:09.546866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:09.547634 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:09.547636 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:09.548049 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:09.548050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:09.548172 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:09.548599 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:23:09.548606 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:23:09.550127 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:23:09.551854 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:23:09.551860 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:23:09.551862 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:23:09.552094 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:23:09.552351 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:23:09.552352 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:23:09.552352 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:23:09.552463 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:23:09.552729 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:23:09.552735 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:23:09.552868 (+ 133us) client_negotiation.cc:769] Sending connection context
0926 06:23:09.552973 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 06:23:09.553146 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":68,"threads_started":1}
I20260926 06:23:09.553840 20649 heartbeater.cc:347] Connected to a master server at 127.4.202.190:41615
I20260926 06:23:09.553948 20649 heartbeater.cc:464] Registering TS with master...
I20260926 06:23:09.554112 20649 heartbeater.cc:511] Master 127.4.202.190:41615 requested a full tablet report, sending...
I20260926 06:23:09.554517 20324 ts_manager.cc:194] Registered new tserver with Master: 515692ec901c470b8b3d3f894cd8bb83 (127.4.202.130:38333)
I20260926 06:23:09.555073 20324 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.130@KRBTEST.COM'} at 127.4.202.130:52847
WARNING: no policy specified for kudu/127.4.202.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
I20260926 06:23:09.574028 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.131:0
--local_ip_for_outbound_sockets=127.4.202.131
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:41615
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:41921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:23:09.653218 20654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:23:09.653406 20654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:23:09.653429 20654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:23:09.654474 20654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:23:09.654506 20654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:23:09.654531 20654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.131
I20260926 06:23:09.655874 20654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:41921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:41615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.131
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:23:09.656039 20654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:23:09.656211 20654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:23:09.661038 20660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:23:09.661186 20654 server_base.cc:1034] running on GCE node
W20260926 06:23:09.661068 20662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:23:09.661043 20659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:23:09.661427 20654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:23:09.661675 20654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:23:09.662850 20654 hybrid_clock.cc:648] HybridClock initialized: now 1790403789662829 us; error 38 us; skew 500 ppm
Sep 26 06:23:09 dist-test-slave-pd0t krb5kdc[20279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403789, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:23:09.672020 20654 init.cc:377] Logged in from keytab as kudu/127.4.202.131@KRBTEST.COM (short username kudu)
I20260926 06:23:09.672492 20654 webserver.cc:460] Webserver started at http://127.4.202.131:39595/ using document root <none> and password file <none>
I20260926 06:23:09.672725 20654 fs_manager.cc:359] Metadata directory not provided
I20260926 06:23:09.672798 20654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:23:09.672945 20654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:23:09.674018 20654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/data/instance:
uuid: "a87d2554f9424d85a88fa4556e64ebfe"
format_stamp: "Formatted at 2026-09-26 06:23:09 on dist-test-slave-pd0t"
server_key: "ac4d2fb15c9faaeca852e9f8848af08d"
server_key_iv: "01a9f6168715205339bbb27fe6a47732"
server_key_version: "encryptionkey@0"
I20260926 06:23:09.674386 20654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance:
uuid: "a87d2554f9424d85a88fa4556e64ebfe"
format_stamp: "Formatted at 2026-09-26 06:23:09 on dist-test-slave-pd0t"
server_key: "ac4d2fb15c9faaeca852e9f8848af08d"
server_key_iv: "01a9f6168715205339bbb27fe6a47732"
server_key_version: "encryptionkey@0"
I20260926 06:23:09.675755 20654 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:23:09.676517 20669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:23:09.676754 20654 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:23:09.676853 20654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/wal
uuid: "a87d2554f9424d85a88fa4556e64ebfe"
format_stamp: "Formatted at 2026-09-26 06:23:09 on dist-test-slave-pd0t"
server_key: "ac4d2fb15c9faaeca852e9f8848af08d"
server_key_iv: "01a9f6168715205339bbb27fe6a47732"
server_key_version: "encryptionkey@0"
I20260926 06:23:09.676939 20654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:23:09.692595 20654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:23:09.694888 20654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:23:09.695052 20654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:23:09.695286 20654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:23:09.695576 20654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:23:09.695628 20654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:23:09.695667 20654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:23:09.695695 20654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:23:09.701538 20654 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.131:37353
I20260926 06:23:09.701591 20782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.131:37353 every 8 connection(s)
I20260926 06:23:09.701885 20654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
Sep 26 06:23:09 dist-test-slave-pd0t krb5kdc[20279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403789, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:23:09.708391 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 20654
I20260926 06:23:09.708601 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance
I20260926 06:23:09.708833 4906 external_mini_cluster.cc:1428] Setting key 8667059b76b580c68278c3d2aea0daa7
I20260926 06:23:09.711627 20363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:09.702598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:51057 (local address 127.4.202.190:41615)
0926 06:23:09.702689 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0926 06:23:09.702691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:23:09.702890 (+ 199us) server_negotiation.cc:407] Connection header received
0926 06:23:09.703359 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:23:09.703360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:23:09.703370 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:23:09.703424 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:23:09.704767 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:09.705263 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:09.706260 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:09.706455 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:09.706519 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:09.708845 (+ 2326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:23:09.708846 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:23:09.708846 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:23:09.708854 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:23:09.710488 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:23:09.710936 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:23:09.710937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:23:09.710937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:23:09.710987 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:23:09.711172 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:23:09.711173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:23:09.711173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:23:09.711290 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:23:09.711338 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 06:23:09.711513 (+ 175us) server_negotiation.cc:299] Negotiation successful
0926 06:23:09.711567 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 06:23:09.711860 20785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:09.702526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.131:51057)
0926 06:23:09.702755 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:09.702805 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 06:23:09.703250 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:23:09.703482 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:23:09.703485 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:23:09.704312 (+ 827us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:23:09.704636 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:09.704644 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:09.705373 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:09.705374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:09.706162 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:09.706164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:09.706538 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:09.706539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:09.706730 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:09.707246 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:23:09.707255 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:23:09.708758 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:23:09.710558 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:23:09.710563 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:23:09.710565 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:23:09.710870 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:23:09.711031 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:23:09.711031 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:23:09.711032 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:23:09.711114 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:23:09.711328 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:23:09.711330 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:23:09.711428 (+ 98us) client_negotiation.cc:769] Sending connection context
0926 06:23:09.711496 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 06:23:09.711602 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":113,"threads_started":1}
I20260926 06:23:09.712293 20783 heartbeater.cc:347] Connected to a master server at 127.4.202.190:41615
I20260926 06:23:09.712381 20783 heartbeater.cc:464] Registering TS with master...
I20260926 06:23:09.712513 20783 heartbeater.cc:511] Master 127.4.202.190:41615 requested a full tablet report, sending...
I20260926 06:23:09.712855 20324 ts_manager.cc:194] Registered new tserver with Master: a87d2554f9424d85a88fa4556e64ebfe (127.4.202.131:37353)
I20260926 06:23:09.713182 20324 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.131@KRBTEST.COM'} at 127.4.202.131:51057
I20260926 06:23:09.720513 4906 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:23:09.727469 20363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:09.721812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39154 (local address 127.4.202.190:41615)
0926 06:23:09.721865 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0926 06:23:09.721867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:23:09.721965 (+ 98us) server_negotiation.cc:407] Connection header received
0926 06:23:09.721998 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:23:09.721999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:23:09.722006 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:23:09.722053 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:23:09.722544 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:09.723034 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:09.723797 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:09.723975 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:09.724029 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:09.725153 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:23:09.725154 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:23:09.725154 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:23:09.725160 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:23:09.726536 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:23:09.726891 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:23:09.726892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:23:09.726892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:23:09.726942 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:23:09.727082 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:23:09.727082 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:23:09.727082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:23:09.727191 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:23:09.727257 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 06:23:09.727358 (+ 101us) server_negotiation.cc:299] Negotiation successful
0926 06:23:09.727413 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 06:23:09.729075 20324 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39154:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:23:09.730674 20324 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:23:09.737023 20796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:09.732967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55238 (local address 127.4.202.131:37353)
0926 06:23:09.733126 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0926 06:23:09.733130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:23:09.733494 (+ 364us) server_negotiation.cc:407] Connection header received
0926 06:23:09.733533 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:23:09.733535 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:23:09.733549 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:23:09.733615 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:23:09.733795 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:09.734820 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:09.736106 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:09.736586 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:09.736680 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:09.736744 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 06:23:09.736908 (+ 164us) server_negotiation.cc:299] Negotiation successful
0926 06:23:09.736945 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":73,"threads_started":1}
I20260926 06:23:09.737016 20792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:09.732772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:37353 (local address 127.0.0.1:55238)
0926 06:23:09.733351 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:09.733359 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 06:23:09.733425 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:23:09.733674 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:23:09.733675 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:23:09.733677 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:23:09.733725 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:09.733726 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:09.734917 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:09.734918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:09.736026 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:09.736028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:09.736687 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:09.736688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:09.736772 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:09.736785 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 06:23:09.736892 (+ 107us) client_negotiation.cc:240] Negotiation successful
0926 06:23:09.736946 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":60,"threads_started":1}
I20260926 06:23:09.737097 20795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:09.732916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40484 (local address 127.4.202.129:45397)
0926 06:23:09.733095 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0926 06:23:09.733099 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:23:09.733110 (+ 11us) server_negotiation.cc:407] Connection header received
0926 06:23:09.733150 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:23:09.733153 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:23:09.733172 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:23:09.733257 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:23:09.733508 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:09.734638 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:09.735894 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:09.736634 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:09.736704 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:09.736757 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 06:23:09.736965 (+ 208us) server_negotiation.cc:299] Negotiation successful
0926 06:23:09.737009 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":80,"threads_started":1}
I20260926 06:23:09.737452 20791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:09.732766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:38333 (local address 127.0.0.1:40580)
0926 06:23:09.733143 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:09.733153 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:23:09.733220 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:23:09.733417 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:23:09.733418 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:23:09.733421 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:23:09.733552 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:09.733554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:09.734470 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:09.734471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:09.735895 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:09.735898 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:09.737199 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:09.737200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:09.737268 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:09.737273 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 06:23:09.737327 (+ 54us) client_negotiation.cc:240] Negotiation successful
0926 06:23:09.737383 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":131,"thread_start_us":66,"threads_started":1}
I20260926 06:23:09.737479 20794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:09.732916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40580 (local address 127.4.202.130:38333)
0926 06:23:09.733095 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0926 06:23:09.733099 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:23:09.733259 (+ 160us) server_negotiation.cc:407] Connection header received
0926 06:23:09.733302 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:23:09.733305 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:23:09.733320 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:23:09.733409 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:23:09.733637 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:09.734368 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:09.735968 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:09.736634 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:09.736704 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:09.736761 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 06:23:09.737361 (+ 600us) server_negotiation.cc:299] Negotiation successful
0926 06:23:09.737406 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":78,"threads_started":1}
I20260926 06:23:09.737746 20717 tablet_service.cc:1467] Processing CreateTablet for tablet e3ee6b3a296e48a4a16a125e0be5e049 (DEFAULT_TABLE table=test-table [id=cff57303b82d4b63a49890c65188d427]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:23:09.738098 20717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3ee6b3a296e48a4a16a125e0be5e049. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:23:09.738245 20793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:09.732761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:45397 (local address 127.0.0.1:40484)
0926 06:23:09.732917 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:09.732953 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 06:23:09.733022 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:23:09.733276 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:23:09.733279 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:23:09.733282 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:23:09.733346 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:09.733348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:09.734760 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:09.734761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:09.735801 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:09.735803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:09.736793 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:09.736795 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:09.736871 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:09.736874 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:23:09.736943 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 06:23:09.736993 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":57,"threads_started":1}
I20260926 06:23:09.738322 20583 tablet_service.cc:1467] Processing CreateTablet for tablet e3ee6b3a296e48a4a16a125e0be5e049 (DEFAULT_TABLE table=test-table [id=cff57303b82d4b63a49890c65188d427]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:23:09.738708 20583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3ee6b3a296e48a4a16a125e0be5e049. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:23:09.738950 20449 tablet_service.cc:1467] Processing CreateTablet for tablet e3ee6b3a296e48a4a16a125e0be5e049 (DEFAULT_TABLE table=test-table [id=cff57303b82d4b63a49890c65188d427]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:23:09.739284 20449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3ee6b3a296e48a4a16a125e0be5e049. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:23:09.741294 20797 tablet_bootstrap.cc:492] T e3ee6b3a296e48a4a16a125e0be5e049 P 515692ec901c470b8b3d3f894cd8bb83: Bootstrap starting.
I20260926 06:23:09.741693 20798 tablet_bootstrap.cc:492] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe: Bootstrap starting.
I20260926 06:23:09.741986 20797 tablet_bootstrap.cc:654] T e3ee6b3a296e48a4a16a125e0be5e049 P 515692ec901c470b8b3d3f894cd8bb83: Neither blocks nor log segments found. Creating new log.
I20260926 06:23:09.742316 20798 tablet_bootstrap.cc:654] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe: Neither blocks nor log segments found. Creating new log.
I20260926 06:23:09.742534 20797 log.cc:824] T e3ee6b3a296e48a4a16a125e0be5e049 P 515692ec901c470b8b3d3f894cd8bb83: Log is configured to *not* fsync() on all Append() calls
I20260926 06:23:09.742698 20799 tablet_bootstrap.cc:492] T e3ee6b3a296e48a4a16a125e0be5e049 P c8900ea32c9e4ca586522252586ec729: Bootstrap starting.
I20260926 06:23:09.742882 20798 log.cc:824] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe: Log is configured to *not* fsync() on all Append() calls
I20260926 06:23:09.743436 20799 tablet_bootstrap.cc:654] T e3ee6b3a296e48a4a16a125e0be5e049 P c8900ea32c9e4ca586522252586ec729: Neither blocks nor log segments found. Creating new log.
I20260926 06:23:09.743443 20797 tablet_bootstrap.cc:492] T e3ee6b3a296e48a4a16a125e0be5e049 P 515692ec901c470b8b3d3f894cd8bb83: No bootstrap required, opened a new log
I20260926 06:23:09.743516 20797 ts_tablet_manager.cc:1397] T e3ee6b3a296e48a4a16a125e0be5e049 P 515692ec901c470b8b3d3f894cd8bb83: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 06:23:09.743643 20798 tablet_bootstrap.cc:492] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe: No bootstrap required, opened a new log
I20260926 06:23:09.743703 20798 ts_tablet_manager.cc:1397] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 06:23:09.743970 20799 log.cc:824] T e3ee6b3a296e48a4a16a125e0be5e049 P c8900ea32c9e4ca586522252586ec729: Log is configured to *not* fsync() on all Append() calls
I20260926 06:23:09.744561 20799 tablet_bootstrap.cc:492] T e3ee6b3a296e48a4a16a125e0be5e049 P c8900ea32c9e4ca586522252586ec729: No bootstrap required, opened a new log
I20260926 06:23:09.744634 20799 ts_tablet_manager.cc:1397] T e3ee6b3a296e48a4a16a125e0be5e049 P c8900ea32c9e4ca586522252586ec729: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:23:09.745460 20798 raft_consensus.cc:348] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8900ea32c9e4ca586522252586ec729" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45397 } } peers { permanent_uuid: "515692ec901c470b8b3d3f894cd8bb83" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38333 } } peers { permanent_uuid: "a87d2554f9424d85a88fa4556e64ebfe" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37353 } }
I20260926 06:23:09.745424 20797 raft_consensus.cc:348] T e3ee6b3a296e48a4a16a125e0be5e049 P 515692ec901c470b8b3d3f894cd8bb83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8900ea32c9e4ca586522252586ec729" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45397 } } peers { permanent_uuid: "515692ec901c470b8b3d3f894cd8bb83" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38333 } } peers { permanent_uuid: "a87d2554f9424d85a88fa4556e64ebfe" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37353 } }
I20260926 06:23:09.745601 20798 raft_consensus.cc:374] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:23:09.745601 20797 raft_consensus.cc:374] T e3ee6b3a296e48a4a16a125e0be5e049 P 515692ec901c470b8b3d3f894cd8bb83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:23:09.745637 20798 raft_consensus.cc:729] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a87d2554f9424d85a88fa4556e64ebfe, State: Initialized, Role: FOLLOWER
I20260926 06:23:09.745637 20797 raft_consensus.cc:729] T e3ee6b3a296e48a4a16a125e0be5e049 P 515692ec901c470b8b3d3f894cd8bb83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 515692ec901c470b8b3d3f894cd8bb83, State: Initialized, Role: FOLLOWER
I20260926 06:23:09.745752 20798 consensus_queue.cc:260] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8900ea32c9e4ca586522252586ec729" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45397 } } peers { permanent_uuid: "515692ec901c470b8b3d3f894cd8bb83" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38333 } } peers { permanent_uuid: "a87d2554f9424d85a88fa4556e64ebfe" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37353 } }
I20260926 06:23:09.745801 20797 consensus_queue.cc:260] T e3ee6b3a296e48a4a16a125e0be5e049 P 515692ec901c470b8b3d3f894cd8bb83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8900ea32c9e4ca586522252586ec729" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45397 } } peers { permanent_uuid: "515692ec901c470b8b3d3f894cd8bb83" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38333 } } peers { permanent_uuid: "a87d2554f9424d85a88fa4556e64ebfe" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37353 } }
I20260926 06:23:09.746055 20797 ts_tablet_manager.cc:1428] T e3ee6b3a296e48a4a16a125e0be5e049 P 515692ec901c470b8b3d3f894cd8bb83: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260926 06:23:09.746055 20798 ts_tablet_manager.cc:1428] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260926 06:23:09.746200 20783 heartbeater.cc:503] Master 127.4.202.190:41615 was elected leader, sending a full tablet report...
I20260926 06:23:09.746166 20799 raft_consensus.cc:348] T e3ee6b3a296e48a4a16a125e0be5e049 P c8900ea32c9e4ca586522252586ec729 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8900ea32c9e4ca586522252586ec729" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45397 } } peers { permanent_uuid: "515692ec901c470b8b3d3f894cd8bb83" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38333 } } peers { permanent_uuid: "a87d2554f9424d85a88fa4556e64ebfe" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37353 } }
I20260926 06:23:09.746335 20649 heartbeater.cc:503] Master 127.4.202.190:41615 was elected leader, sending a full tablet report...
I20260926 06:23:09.746344 20799 raft_consensus.cc:374] T e3ee6b3a296e48a4a16a125e0be5e049 P c8900ea32c9e4ca586522252586ec729 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:23:09.746377 20799 raft_consensus.cc:729] T e3ee6b3a296e48a4a16a125e0be5e049 P c8900ea32c9e4ca586522252586ec729 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8900ea32c9e4ca586522252586ec729, State: Initialized, Role: FOLLOWER
I20260926 06:23:09.746501 20799 consensus_queue.cc:260] T e3ee6b3a296e48a4a16a125e0be5e049 P c8900ea32c9e4ca586522252586ec729 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8900ea32c9e4ca586522252586ec729" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45397 } } peers { permanent_uuid: "515692ec901c470b8b3d3f894cd8bb83" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38333 } } peers { permanent_uuid: "a87d2554f9424d85a88fa4556e64ebfe" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37353 } }
I20260926 06:23:09.746779 20799 ts_tablet_manager.cc:1428] T e3ee6b3a296e48a4a16a125e0be5e049 P c8900ea32c9e4ca586522252586ec729: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:23:09.746918 20515 heartbeater.cc:503] Master 127.4.202.190:41615 was elected leader, sending a full tablet report...
W20260926 06:23:09.794277 20650 tablet.cc:2369] T e3ee6b3a296e48a4a16a125e0be5e049 P 515692ec901c470b8b3d3f894cd8bb83: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 06:23:09.894229 20516 tablet.cc:2369] T e3ee6b3a296e48a4a16a125e0be5e049 P c8900ea32c9e4ca586522252586ec729: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 06:23:09.952416 20784 tablet.cc:2369] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:23:10.014492 20803 raft_consensus.cc:482] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:23:10.014655 20803 raft_consensus.cc:504] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8900ea32c9e4ca586522252586ec729" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45397 } } peers { permanent_uuid: "515692ec901c470b8b3d3f894cd8bb83" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38333 } } peers { permanent_uuid: "a87d2554f9424d85a88fa4556e64ebfe" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37353 } }
I20260926 06:23:10.015077 20803 leader_election.cc:288] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8900ea32c9e4ca586522252586ec729 (127.4.202.129:45397), 515692ec901c470b8b3d3f894cd8bb83 (127.4.202.130:38333)
I20260926 06:23:10.018941 20794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:10.015377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:50931 (local address 127.4.202.130:38333)
0926 06:23:10.015422 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 06:23:10.015424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:23:10.015433 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:23:10.015462 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:23:10.015464 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:23:10.015473 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:23:10.015531 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:23:10.015731 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:10.016492 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:10.017969 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:10.018544 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:10.018660 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:10.018669 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 06:23:10.018835 (+ 166us) server_negotiation.cc:299] Negotiation successful
0926 06:23:10.018875 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 06:23:10.018941 20795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:10.015349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:56867 (local address 127.4.202.129:45397)
0926 06:23:10.015420 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0926 06:23:10.015423 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:23:10.015640 (+ 217us) server_negotiation.cc:407] Connection header received
0926 06:23:10.015666 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:23:10.015667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:23:10.015694 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:23:10.015750 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:23:10.015870 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:10.016601 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:10.017964 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:10.018544 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:10.018659 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:10.018669 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 06:23:10.018836 (+ 167us) server_negotiation.cc:299] Negotiation successful
0926 06:23:10.018875 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 06:23:10.018975 20785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:10.015244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:38333 (local address 127.4.202.131:50931)
0926 06:23:10.015293 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:10.015322 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 06:23:10.015370 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:23:10.015560 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:23:10.015561 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:23:10.015563 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:23:10.015638 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:10.015640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:10.016577 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:10.016578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:10.017870 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:10.017872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:10.018659 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:10.018660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:10.018746 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:10.018750 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:23:10.018814 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 06:23:10.018860 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21,"mutex_wait_us":45}
I20260926 06:23:10.018981 20806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:10.015251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:45397 (local address 127.4.202.131:56867)
0926 06:23:10.015557 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:10.015567 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:23:10.015622 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:23:10.015751 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:23:10.015751 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:23:10.015753 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:23:10.015809 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:10.015810 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:10.016666 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:10.016667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:10.017869 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:10.017872 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:10.018660 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:10.018660 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:10.018762 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:10.018765 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:23:10.018818 (+ 53us) client_negotiation.cc:240] Negotiation successful
0926 06:23:10.018860 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":159,"thread_start_us":112,"threads_started":1}
I20260926 06:23:10.019407 20469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3ee6b3a296e48a4a16a125e0be5e049" candidate_uuid: "a87d2554f9424d85a88fa4556e64ebfe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8900ea32c9e4ca586522252586ec729" is_pre_election: true
I20260926 06:23:10.019505 20603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3ee6b3a296e48a4a16a125e0be5e049" candidate_uuid: "a87d2554f9424d85a88fa4556e64ebfe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "515692ec901c470b8b3d3f894cd8bb83" is_pre_election: true
I20260926 06:23:10.019574 20469 raft_consensus.cc:2454] T e3ee6b3a296e48a4a16a125e0be5e049 P c8900ea32c9e4ca586522252586ec729 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a87d2554f9424d85a88fa4556e64ebfe in term 0.
I20260926 06:23:10.019683 20603 raft_consensus.cc:2454] T e3ee6b3a296e48a4a16a125e0be5e049 P 515692ec901c470b8b3d3f894cd8bb83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a87d2554f9424d85a88fa4556e64ebfe in term 0.
I20260926 06:23:10.019814 20672 leader_election.cc:302] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a87d2554f9424d85a88fa4556e64ebfe, c8900ea32c9e4ca586522252586ec729; no voters:
I20260926 06:23:10.020006 20803 raft_consensus.cc:2781] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:23:10.020067 20803 raft_consensus.cc:482] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:23:10.020084 20803 raft_consensus.cc:3037] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:23:10.020865 20803 raft_consensus.cc:504] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8900ea32c9e4ca586522252586ec729" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45397 } } peers { permanent_uuid: "515692ec901c470b8b3d3f894cd8bb83" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38333 } } peers { permanent_uuid: "a87d2554f9424d85a88fa4556e64ebfe" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37353 } }
I20260926 06:23:10.021050 20803 leader_election.cc:288] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe [CANDIDATE]: Term 1 election: Requested vote from peers c8900ea32c9e4ca586522252586ec729 (127.4.202.129:45397), 515692ec901c470b8b3d3f894cd8bb83 (127.4.202.130:38333)
I20260926 06:23:10.021191 20603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3ee6b3a296e48a4a16a125e0be5e049" candidate_uuid: "a87d2554f9424d85a88fa4556e64ebfe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "515692ec901c470b8b3d3f894cd8bb83"
I20260926 06:23:10.021199 20469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3ee6b3a296e48a4a16a125e0be5e049" candidate_uuid: "a87d2554f9424d85a88fa4556e64ebfe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8900ea32c9e4ca586522252586ec729"
I20260926 06:23:10.021270 20603 raft_consensus.cc:3037] T e3ee6b3a296e48a4a16a125e0be5e049 P 515692ec901c470b8b3d3f894cd8bb83 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:23:10.021291 20469 raft_consensus.cc:3037] T e3ee6b3a296e48a4a16a125e0be5e049 P c8900ea32c9e4ca586522252586ec729 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:23:10.021909 20603 raft_consensus.cc:2454] T e3ee6b3a296e48a4a16a125e0be5e049 P 515692ec901c470b8b3d3f894cd8bb83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a87d2554f9424d85a88fa4556e64ebfe in term 1.
I20260926 06:23:10.022109 20671 leader_election.cc:302] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 515692ec901c470b8b3d3f894cd8bb83, a87d2554f9424d85a88fa4556e64ebfe; no voters:
I20260926 06:23:10.022125 20469 raft_consensus.cc:2454] T e3ee6b3a296e48a4a16a125e0be5e049 P c8900ea32c9e4ca586522252586ec729 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a87d2554f9424d85a88fa4556e64ebfe in term 1.
I20260926 06:23:10.022238 20803 raft_consensus.cc:2781] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:23:10.022385 20803 raft_consensus.cc:686] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe [term 1 LEADER]: Becoming Leader. State: Replica: a87d2554f9424d85a88fa4556e64ebfe, State: Running, Role: LEADER
I20260926 06:23:10.022476 20803 consensus_queue.cc:237] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8900ea32c9e4ca586522252586ec729" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45397 } } peers { permanent_uuid: "515692ec901c470b8b3d3f894cd8bb83" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38333 } } peers { permanent_uuid: "a87d2554f9424d85a88fa4556e64ebfe" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37353 } }
I20260926 06:23:10.023346 20323 catalog_manager.cc:5495] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe reported cstate change: term changed from 0 to 1, leader changed from <none> to a87d2554f9424d85a88fa4556e64ebfe (127.4.202.131). New cstate: current_term: 1 leader_uuid: "a87d2554f9424d85a88fa4556e64ebfe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8900ea32c9e4ca586522252586ec729" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "515692ec901c470b8b3d3f894cd8bb83" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a87d2554f9424d85a88fa4556e64ebfe" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37353 } health_report { overall_health: HEALTHY } } }
I20260926 06:23:10.089632 20796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:10.087362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55240 (local address 127.4.202.131:37353)
0926 06:23:10.087425 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 06:23:10.087427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:23:10.087436 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:23:10.087478 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:23:10.087478 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:23:10.087495 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:23:10.087555 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:23:10.087682 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:10.088134 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:10.088891 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:10.089078 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:10.089131 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:10.089297 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:23:10.089389 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:23:10.089449 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 06:23:10.089544 (+ 95us) server_negotiation.cc:299] Negotiation successful
0926 06:23:10.089562 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 06:23:10.091455 20469 raft_consensus.cc:1261] T e3ee6b3a296e48a4a16a125e0be5e049 P c8900ea32c9e4ca586522252586ec729 [term 1 FOLLOWER]: Refusing update from remote peer a87d2554f9424d85a88fa4556e64ebfe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:23:10.091475 20603 raft_consensus.cc:1261] T e3ee6b3a296e48a4a16a125e0be5e049 P 515692ec901c470b8b3d3f894cd8bb83 [term 1 FOLLOWER]: Refusing update from remote peer a87d2554f9424d85a88fa4556e64ebfe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:23:10.091849 20807 consensus_queue.cc:1035] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe [LEADER]: Connected to new peer: Peer: permanent_uuid: "515692ec901c470b8b3d3f894cd8bb83" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:23:10.091986 20803 consensus_queue.cc:1035] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8900ea32c9e4ca586522252586ec729" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:23:10.094367 20812 mvcc.cc:205] Tried to move back new op lower bound from 7333493924211908608 to 7333493923932741632. Current Snapshot: MvccSnapshot[applied={T|T < 7333493924211908608}]
I20260926 06:23:10.095283 20811 mvcc.cc:205] Tried to move back new op lower bound from 7333493924211908608 to 7333493923932741632. Current Snapshot: MvccSnapshot[applied={T|T < 7333493924211908608 or (T in {7333493924211908608})}]
I20260926 06:23:10.096508 20323 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39154:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:23:10.096612 20323 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39154:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:23:10.097434 20323 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099: Sending DeleteTablet for 3 replicas of tablet e3ee6b3a296e48a4a16a125e0be5e049
I20260926 06:23:10.097776 20717 tablet_service.cc:1514] Processing DeleteTablet for tablet e3ee6b3a296e48a4a16a125e0be5e049 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:23:10 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:55238
I20260926 06:23:10.097787 20583 tablet_service.cc:1514] Processing DeleteTablet for tablet e3ee6b3a296e48a4a16a125e0be5e049 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:23:10 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:40580
I20260926 06:23:10.097810 20449 tablet_service.cc:1514] Processing DeleteTablet for tablet e3ee6b3a296e48a4a16a125e0be5e049 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:23:10 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:40484
I20260926 06:23:10.098061 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 20386
I20260926 06:23:10.098076 20822 tablet_replica.cc:330] T e3ee6b3a296e48a4a16a125e0be5e049 P 515692ec901c470b8b3d3f894cd8bb83: stopping tablet replica
I20260926 06:23:10.098974 20822 raft_consensus.cc:2229] T e3ee6b3a296e48a4a16a125e0be5e049 P 515692ec901c470b8b3d3f894cd8bb83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:23:10.099162 20822 raft_consensus.cc:2258] T e3ee6b3a296e48a4a16a125e0be5e049 P 515692ec901c470b8b3d3f894cd8bb83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:23:10.099530 20820 tablet_replica.cc:330] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe: stopping tablet replica
I20260926 06:23:10.099625 20820 raft_consensus.cc:2229] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:23:10.099721 20820 raft_consensus.cc:2258] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:23:10.099977 20822 ts_tablet_manager.cc:1905] T e3ee6b3a296e48a4a16a125e0be5e049 P 515692ec901c470b8b3d3f894cd8bb83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:23:10.101985 20822 ts_tablet_manager.cc:1918] T e3ee6b3a296e48a4a16a125e0be5e049 P 515692ec901c470b8b3d3f894cd8bb83: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:23:10.102075 20822 log.cc:1196] T e3ee6b3a296e48a4a16a125e0be5e049 P 515692ec901c470b8b3d3f894cd8bb83: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/wal/wals/e3ee6b3a296e48a4a16a125e0be5e049
I20260926 06:23:10.102317 20822 ts_tablet_manager.cc:1939] T e3ee6b3a296e48a4a16a125e0be5e049 P 515692ec901c470b8b3d3f894cd8bb83: Deleting consensus metadata
W20260926 06:23:10.104197 20310 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.202.129:45397 (error 108)
I20260926 06:23:10.104492 20309 catalog_manager.cc:4841] TS 515692ec901c470b8b3d3f894cd8bb83 (127.4.202.130:38333): tablet e3ee6b3a296e48a4a16a125e0be5e049 (table test-table [id=cff57303b82d4b63a49890c65188d427]) successfully deleted
I20260926 06:23:10.104660 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 20520
I20260926 06:23:10.105197 20793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:10.104608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:45397 (local address 127.0.0.1:40498)
0926 06:23:10.105105 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:10.105168 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.129:45397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474}
W20260926 06:23:10.105365 20310 catalog_manager.cc:4584] TS c8900ea32c9e4ca586522252586ec729 (127.4.202.129:45397): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e3ee6b3a296e48a4a16a125e0be5e049: Network error: Client connection negotiation failed: client connection to 127.4.202.129:45397: connect: Connection refused (error 111)
I20260926 06:23:10.110196 20820 ts_tablet_manager.cc:1905] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:23:10.110234 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 20654
I20260926 06:23:10.114367 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 20292
I20260926 06:23:10.114500 20793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:10.114357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:37353 (local address 127.0.0.1:55246)
0926 06:23:10.114432 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:10.114463 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.131:37353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260926 06:23:30.119621 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:41615
--webserver_interface=127.4.202.190
--webserver_port=46051
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:41921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:41615
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:23:30.199343 20824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:23:30.199518 20824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:23:30.199538 20824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:23:30.200671 20824 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 06:23:30.200711 20824 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 06:23:30.200724 20824 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 06:23:30.200735 20824 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:23:30.200770 20824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:23:30.200817 20824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:23:30.200840 20824 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:23:30.200865 20824 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:23:30.202353 20824 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:41921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:41615
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:41615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=46051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:23:30.202620 20824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:23:30.202858 20824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:23:30.210980 20830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:23:30.210949 20833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:23:30.211172 20831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:23:30.211231 20824 server_base.cc:1034] running on GCE node
I20260926 06:23:30.211522 20824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:23:30.211791 20824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:23:30.212963 20824 hybrid_clock.cc:648] HybridClock initialized: now 1790403810212953 us; error 45 us; skew 500 ppm
Sep 26 06:23:30 dist-test-slave-pd0t krb5kdc[20279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403810, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:23:30.215523 20824 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:23:30.215993 20824 webserver.cc:460] Webserver started at http://127.4.202.190:46051/ using document root <none> and password file <none>
I20260926 06:23:30.216236 20824 fs_manager.cc:359] Metadata directory not provided
I20260926 06:23:30.216318 20824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:23:30.217654 20824 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:23:30.218389 20840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:23:30.218619 20824 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:23:30.218726 20824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "90b703134eac456597feb5ca43537099"
format_stamp: "Formatted at 2026-09-26 06:23:09 on dist-test-slave-pd0t"
server_key: "931169c4c3d8c12ba5b4407b7abb25a4"
server_key_iv: "3d8f79484670ed072ea8083ad0646833"
server_key_version: "encryptionkey@0"
I20260926 06:23:30.219091 20824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:23:30.239029 20824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:23:30.241276 20824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:23:30.241432 20824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:23:30.245239 20824 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:41615
I20260926 06:23:30.245272 20892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:41615 every 8 connection(s)
I20260926 06:23:30.245647 20824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:23:30.247108 20893 sys_catalog.cc:263] Verifying existing consensus state
I20260926 06:23:30.247677 20893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099: Bootstrap starting.
I20260926 06:23:30.254709 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 20824
I20260926 06:23:30.254997 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.129:45397
--local_ip_for_outbound_sockets=127.4.202.129
--tserver_master_addrs=127.4.202.190:41615
--webserver_port=35235
--webserver_interface=127.4.202.129
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:41921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:23:30.255839 20893 log.cc:824] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099: Log is configured to *not* fsync() on all Append() calls
I20260926 06:23:30.257925 20893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 06:23:30.258215 20893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099: Bootstrap complete.
I20260926 06:23:30.260371 20893 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90b703134eac456597feb5ca43537099" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 41615 } }
I20260926 06:23:30.260597 20893 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90b703134eac456597feb5ca43537099, State: Initialized, Role: FOLLOWER
I20260926 06:23:30.260803 20893 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90b703134eac456597feb5ca43537099" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 41615 } }
I20260926 06:23:30.260900 20893 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:23:30.260959 20893 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:23:30.261009 20893 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [term 1 FOLLOWER]: Advancing to term 2
I20260926 06:23:30.261983 20893 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90b703134eac456597feb5ca43537099" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 41615 } }
I20260926 06:23:30.262105 20893 leader_election.cc:302] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90b703134eac456597feb5ca43537099; no voters:
I20260926 06:23:30.262298 20893 leader_election.cc:288] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 06:23:30.262336 20897 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 06:23:30.262444 20897 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [term 2 LEADER]: Becoming Leader. State: Replica: 90b703134eac456597feb5ca43537099, State: Running, Role: LEADER
I20260926 06:23:30.262573 20893 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:23:30.262638 20897 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90b703134eac456597feb5ca43537099" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 41615 } }
I20260926 06:23:30.263007 20898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "90b703134eac456597feb5ca43537099" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90b703134eac456597feb5ca43537099" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 41615 } } }
I20260926 06:23:30.263028 20899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90b703134eac456597feb5ca43537099. Latest consensus state: current_term: 2 leader_uuid: "90b703134eac456597feb5ca43537099" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90b703134eac456597feb5ca43537099" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 41615 } } }
I20260926 06:23:30.263100 20898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [sys.catalog]: This master's current role is: LEADER
I20260926 06:23:30.263123 20899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [sys.catalog]: This master's current role is: LEADER
I20260926 06:23:30.263329 20904 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:23:30.264184 20904 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:23:30.264328 20904 catalog_manager.cc:1259] Loaded cluster ID: 286568cd652242be8b3d784b16bff7a8
I20260926 06:23:30.264648 20904 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:23:30.265810 20904 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:23:30.266031 20904 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099: Loaded TSK: 0
I20260926 06:23:30.276713 20904 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099: Generated new TSK 1
I20260926 06:23:30.276783 20904 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 06:23:30.336328 20895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:23:30.336517 20895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:23:30.336540 20895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:23:30.337663 20895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:23:30.337702 20895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:23:30.337728 20895 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.129
I20260926 06:23:30.339174 20895 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:41921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.129:45397
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.202.129
--webserver_port=35235
--tserver_master_addrs=127.4.202.190:41615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.129
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:23:30.339448 20895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:23:30.339675 20895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:23:30.344532 20921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:23:30.344547 20923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:23:30.344566 20920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:23:30.344937 20895 server_base.cc:1034] running on GCE node
I20260926 06:23:30.345121 20895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:23:30.345391 20895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:23:30.346603 20895 hybrid_clock.cc:648] HybridClock initialized: now 1790403810346562 us; error 33 us; skew 500 ppm
Sep 26 06:23:30 dist-test-slave-pd0t krb5kdc[20279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403810, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:23:30.349431 20895 init.cc:377] Logged in from keytab as kudu/127.4.202.129@KRBTEST.COM (short username kudu)
I20260926 06:23:30.349881 20895 webserver.cc:460] Webserver started at http://127.4.202.129:35235/ using document root <none> and password file <none>
I20260926 06:23:30.350099 20895 fs_manager.cc:359] Metadata directory not provided
I20260926 06:23:30.350181 20895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:23:30.351572 20895 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:23:30.352313 20930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:23:30.352550 20895 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:23:30.352653 20895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/wal
uuid: "c8900ea32c9e4ca586522252586ec729"
format_stamp: "Formatted at 2026-09-26 06:23:09 on dist-test-slave-pd0t"
server_key: "25409706138cc6fc0dddab57397c20fa"
server_key_iv: "b5ea3da294d48df3e8520d03197b02ce"
server_key_version: "encryptionkey@0"
I20260926 06:23:30.353044 20895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:23:30.371526 20895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:23:30.373919 20895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:23:30.374073 20895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:23:30.374303 20895 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:23:30.374797 20937 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 06:23:30.376219 20895 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 06:23:30.376286 20895 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260926 06:23:30.376319 20895 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 06:23:30.377142 20895 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 06:23:30.377188 20895 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260926 06:23:30.377281 20937 tablet_bootstrap.cc:492] T e3ee6b3a296e48a4a16a125e0be5e049 P c8900ea32c9e4ca586522252586ec729: Bootstrap starting.
I20260926 06:23:30.384255 20895 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.129:45397
I20260926 06:23:30.384718 20895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
I20260926 06:23:30.386739 21044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.129:45397 every 8 connection(s)
I20260926 06:23:30.387908 20937 log.cc:824] T e3ee6b3a296e48a4a16a125e0be5e049 P c8900ea32c9e4ca586522252586ec729: Log is configured to *not* fsync() on all Append() calls
I20260926 06:23:30.389813 20937 tablet_bootstrap.cc:492] T e3ee6b3a296e48a4a16a125e0be5e049 P c8900ea32c9e4ca586522252586ec729: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 06:23:30.390028 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 20895
I20260926 06:23:30.390190 20937 tablet_bootstrap.cc:492] T e3ee6b3a296e48a4a16a125e0be5e049 P c8900ea32c9e4ca586522252586ec729: Bootstrap complete.
I20260926 06:23:30.390302 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.130:38333
--local_ip_for_outbound_sockets=127.4.202.130
--tserver_master_addrs=127.4.202.190:41615
--webserver_port=41613
--webserver_interface=127.4.202.130
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:41921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:23:30.391358 20937 ts_tablet_manager.cc:1397] T e3ee6b3a296e48a4a16a125e0be5e049 P c8900ea32c9e4ca586522252586ec729: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.006s
I20260926 06:23:30.392967 20937 raft_consensus.cc:348] T e3ee6b3a296e48a4a16a125e0be5e049 P c8900ea32c9e4ca586522252586ec729 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8900ea32c9e4ca586522252586ec729" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45397 } } peers { permanent_uuid: "515692ec901c470b8b3d3f894cd8bb83" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38333 } } peers { permanent_uuid: "a87d2554f9424d85a88fa4556e64ebfe" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37353 } }
I20260926 06:23:30.393345 20937 raft_consensus.cc:729] T e3ee6b3a296e48a4a16a125e0be5e049 P c8900ea32c9e4ca586522252586ec729 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8900ea32c9e4ca586522252586ec729, State: Initialized, Role: FOLLOWER
I20260926 06:23:30.395401 20937 consensus_queue.cc:260] T e3ee6b3a296e48a4a16a125e0be5e049 P c8900ea32c9e4ca586522252586ec729 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8900ea32c9e4ca586522252586ec729" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45397 } } peers { permanent_uuid: "515692ec901c470b8b3d3f894cd8bb83" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38333 } } peers { permanent_uuid: "a87d2554f9424d85a88fa4556e64ebfe" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37353 } }
I20260926 06:23:30.395800 20937 ts_tablet_manager.cc:1428] T e3ee6b3a296e48a4a16a125e0be5e049 P c8900ea32c9e4ca586522252586ec729: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
Sep 26 06:23:30 dist-test-slave-pd0t krb5kdc[20279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403810, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:23:30.402994 21047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:30.387181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.129:53761)
0926 06:23:30.390090 (+ 2909us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:30.390135 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 06:23:30.390829 (+ 694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:23:30.391393 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:23:30.391396 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:23:30.391745 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:23:30.392013 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:30.392020 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:30.393775 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:30.393776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:30.395154 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:30.395158 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:30.396105 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:30.396106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:30.396238 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:30.396892 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:23:30.396911 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:23:30.398852 (+ 1941us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:23:30.401053 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:23:30.401061 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:23:30.401063 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:23:30.401413 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:23:30.401733 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:23:30.401734 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:23:30.401735 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:23:30.401865 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:23:30.402343 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:23:30.402350 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:23:30.402496 (+ 146us) client_negotiation.cc:769] Sending connection context
0926 06:23:30.402619 (+ 123us) client_negotiation.cc:240] Negotiation successful
0926 06:23:30.402799 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2865,"thread_start_us":103,"threads_started":1}
I20260926 06:23:30.403002 21048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:30.387340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:53761 (local address 127.4.202.190:41615)
0926 06:23:30.387525 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0926 06:23:30.387533 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 06:23:30.390240 (+ 2707us) server_negotiation.cc:407] Connection header received
0926 06:23:30.391053 (+ 813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:23:30.391061 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:23:30.391116 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:23:30.391216 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:23:30.392208 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:30.393642 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:30.395252 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:30.395996 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:30.396122 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:30.398961 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:23:30.398965 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:23:30.398966 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:23:30.398978 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:23:30.400959 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:23:30.401518 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:23:30.401521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:23:30.401522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:23:30.401639 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:23:30.401953 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:23:30.401954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:23:30.401954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:23:30.402245 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:23:30.402347 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 06:23:30.402713 (+ 366us) server_negotiation.cc:299] Negotiation successful
0926 06:23:30.402840 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":97,"threads_started":1}
I20260926 06:23:30.404227 21045 heartbeater.cc:347] Connected to a master server at 127.4.202.190:41615
I20260926 06:23:30.404320 21045 heartbeater.cc:464] Registering TS with master...
I20260926 06:23:30.404495 21045 heartbeater.cc:511] Master 127.4.202.190:41615 requested a full tablet report, sending...
I20260926 06:23:30.405239 20857 ts_manager.cc:194] Registered new tserver with Master: c8900ea32c9e4ca586522252586ec729 (127.4.202.129:45397)
I20260926 06:23:30.406374 20857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.129@KRBTEST.COM'} at 127.4.202.129:53761
I20260926 06:23:30.407266 21045 heartbeater.cc:503] Master 127.4.202.190:41615 was elected leader, sending a full tablet report...
Sep 26 06:23:30 dist-test-slave-pd0t krb5kdc[20279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403810, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for kudu/127.4.202.129@KRBTEST.COM
I20260926 06:23:30.413925 21053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:30.405902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55116 (local address 127.4.202.129:45397)
0926 06:23:30.406236 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0926 06:23:30.406239 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:23:30.406246 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:23:30.406269 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:23:30.406270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:23:30.406280 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:23:30.406354 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:23:30.406988 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:30.408078 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:30.408804 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:30.409228 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:30.409299 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:30.411317 (+ 2018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:23:30.411323 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:23:30.411323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:23:30.411329 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:23:30.412729 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:23:30.413183 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:23:30.413185 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:23:30.413185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:23:30.413236 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:23:30.413428 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:23:30.413429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:23:30.413429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:23:30.413553 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:23:30.413612 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 06:23:30.413797 (+ 185us) server_negotiation.cc:299] Negotiation successful
0926 06:23:30.413867 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":67,"threads_started":1}
I20260926 06:23:30.413966 21052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:30.405849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:45397 (local address 127.0.0.1:55116)
0926 06:23:30.406063 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:30.406088 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 06:23:30.406138 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:23:30.406369 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:23:30.406374 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:23:30.406837 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:23:30.406888 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:30.406890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:30.408167 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:30.408168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:30.408725 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:30.408727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:30.409302 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:30.409303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:30.409351 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:30.409775 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:23:30.409778 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:23:30.411230 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:23:30.412810 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:23:30.412817 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:23:30.412819 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:23:30.413080 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:23:30.413302 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:23:30.413302 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:23:30.413303 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:23:30.413368 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:23:30.413615 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:23:30.413622 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:23:30.413677 (+ 55us) client_negotiation.cc:769] Sending connection context
0926 06:23:30.413763 (+ 86us) client_negotiation.cc:240] Negotiation successful
0926 06:23:30.413879 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":82,"threads_started":1}
I20260926 06:23:30.414362 20979 tablet_service.cc:1514] Processing DeleteTablet for tablet e3ee6b3a296e48a4a16a125e0be5e049 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:23:10 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:55116
I20260926 06:23:30.414964 21054 tablet_replica.cc:330] stopping tablet replica
I20260926 06:23:30.415077 21054 raft_consensus.cc:2229] T e3ee6b3a296e48a4a16a125e0be5e049 P c8900ea32c9e4ca586522252586ec729 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:23:30.415175 21054 raft_consensus.cc:2258] T e3ee6b3a296e48a4a16a125e0be5e049 P c8900ea32c9e4ca586522252586ec729 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:23:30.415486 21054 ts_tablet_manager.cc:1905] T e3ee6b3a296e48a4a16a125e0be5e049 P c8900ea32c9e4ca586522252586ec729: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:23:30.417810 21054 ts_tablet_manager.cc:1918] T e3ee6b3a296e48a4a16a125e0be5e049 P c8900ea32c9e4ca586522252586ec729: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:23:30.417905 21054 log.cc:1196] T e3ee6b3a296e48a4a16a125e0be5e049 P c8900ea32c9e4ca586522252586ec729: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/wal/wals/e3ee6b3a296e48a4a16a125e0be5e049
I20260926 06:23:30.418113 21054 ts_tablet_manager.cc:1939] T e3ee6b3a296e48a4a16a125e0be5e049 P c8900ea32c9e4ca586522252586ec729: Deleting consensus metadata
I20260926 06:23:30.418628 20843 catalog_manager.cc:4841] TS c8900ea32c9e4ca586522252586ec729 (127.4.202.129:45397): tablet e3ee6b3a296e48a4a16a125e0be5e049 (table test-table [id=cff57303b82d4b63a49890c65188d427]) successfully deleted
W20260926 06:23:30.476868 21050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:23:30.477046 21050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:23:30.477069 21050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:23:30.478178 21050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:23:30.478217 21050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:23:30.478242 21050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.130
I20260926 06:23:30.479642 21050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:41921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.130:38333
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.202.130
--webserver_port=41613
--tserver_master_addrs=127.4.202.190:41615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.130
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:23:30.479874 21050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:23:30.480085 21050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:23:30.484862 21060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:23:30.484855 21059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:23:30.484980 21050 server_base.cc:1034] running on GCE node
W20260926 06:23:30.484855 21062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:23:30.485265 21050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:23:30.485445 21050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:23:30.486654 21050 hybrid_clock.cc:648] HybridClock initialized: now 1790403810486627 us; error 34 us; skew 500 ppm
Sep 26 06:23:30 dist-test-slave-pd0t krb5kdc[20279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403810, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:23:30.489223 21050 init.cc:377] Logged in from keytab as kudu/127.4.202.130@KRBTEST.COM (short username kudu)
I20260926 06:23:30.489650 21050 webserver.cc:460] Webserver started at http://127.4.202.130:41613/ using document root <none> and password file <none>
I20260926 06:23:30.489856 21050 fs_manager.cc:359] Metadata directory not provided
I20260926 06:23:30.489908 21050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:23:30.491364 21050 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260926 06:23:30.492028 21069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:23:30.492264 21050 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:23:30.492370 21050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/wal
uuid: "515692ec901c470b8b3d3f894cd8bb83"
format_stamp: "Formatted at 2026-09-26 06:23:09 on dist-test-slave-pd0t"
server_key: "30cc733beab913f1436d6311c50c8000"
server_key_iv: "8c8a47d736497e90060cd9acd6d59f6c"
server_key_version: "encryptionkey@0"
I20260926 06:23:30.492753 21050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:23:30.511204 21050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:23:30.513652 21050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:23:30.513821 21050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:23:30.514055 21050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:23:30.514400 21050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:23:30.514458 21050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:23:30.514501 21050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:23:30.514532 21050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:23:30.520818 21050 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.130:38333
I20260926 06:23:30.520865 21182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.130:38333 every 8 connection(s)
I20260926 06:23:30.521262 21050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
I20260926 06:23:30.526324 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 21050
I20260926 06:23:30.526670 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.131:37353
--local_ip_for_outbound_sockets=127.4.202.131
--tserver_master_addrs=127.4.202.190:41615
--webserver_port=39595
--webserver_interface=127.4.202.131
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:41921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 06:23:30 dist-test-slave-pd0t krb5kdc[20279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403810, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:23:30.531219 21048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:30.521981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:38345 (local address 127.4.202.190:41615)
0926 06:23:30.522045 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 06:23:30.522047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:23:30.522298 (+ 251us) server_negotiation.cc:407] Connection header received
0926 06:23:30.522851 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:23:30.522852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:23:30.522866 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:23:30.522923 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:23:30.523840 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:30.524376 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:30.525279 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:30.525477 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:30.525531 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:30.527983 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:23:30.527985 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:23:30.527986 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:23:30.527994 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:23:30.529716 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:23:30.530140 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:23:30.530141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:23:30.530141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:23:30.530204 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:23:30.530467 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:23:30.530468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:23:30.530469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:23:30.530631 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:23:30.530710 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 06:23:30.531011 (+ 301us) server_negotiation.cc:299] Negotiation successful
0926 06:23:30.531120 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 06:23:30.531427 21185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:30.521985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.130:38345)
0926 06:23:30.522184 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:30.522232 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 06:23:30.522744 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:23:30.522973 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:23:30.522977 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:23:30.523576 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:23:30.523755 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:30.523760 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:30.524455 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:30.524456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:30.525205 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:30.525207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:30.525575 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:30.525576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:30.525695 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:30.526157 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:23:30.526168 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:23:30.527880 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:23:30.529819 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:23:30.529827 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:23:30.529829 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:23:30.530056 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:23:30.530276 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:23:30.530277 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:23:30.530278 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:23:30.530393 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:23:30.530691 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:23:30.530698 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:23:30.530869 (+ 171us) client_negotiation.cc:769] Sending connection context
0926 06:23:30.530988 (+ 119us) client_negotiation.cc:240] Negotiation successful
0926 06:23:30.531163 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":93,"threads_started":1}
I20260926 06:23:30.531940 21183 heartbeater.cc:347] Connected to a master server at 127.4.202.190:41615
I20260926 06:23:30.532056 21183 heartbeater.cc:464] Registering TS with master...
I20260926 06:23:30.532284 21183 heartbeater.cc:511] Master 127.4.202.190:41615 requested a full tablet report, sending...
I20260926 06:23:30.532723 20857 ts_manager.cc:194] Registered new tserver with Master: 515692ec901c470b8b3d3f894cd8bb83 (127.4.202.130:38333)
I20260926 06:23:30.533345 20857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.130@KRBTEST.COM'} at 127.4.202.130:38345
W20260926 06:23:30.613696 21186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:23:30.613901 21186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:23:30.613924 21186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:23:30.615020 21186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:23:30.615056 21186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:23:30.615082 21186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.131
I20260926 06:23:30.616434 21186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:41921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.131:37353
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.202.131
--webserver_port=39595
--tserver_master_addrs=127.4.202.190:41615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.131
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:23:30.616641 21186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:23:30.616797 21186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:23:30.621536 21194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:23:30.621547 21192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:23:30.621555 21191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:23:30.621789 21186 server_base.cc:1034] running on GCE node
I20260926 06:23:30.622071 21186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:23:30.622370 21186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:23:30.623518 21186 hybrid_clock.cc:648] HybridClock initialized: now 1790403810623505 us; error 28 us; skew 500 ppm
Sep 26 06:23:30 dist-test-slave-pd0t krb5kdc[20279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403810, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:23:30.625862 21186 init.cc:377] Logged in from keytab as kudu/127.4.202.131@KRBTEST.COM (short username kudu)
I20260926 06:23:30.626310 21186 webserver.cc:460] Webserver started at http://127.4.202.131:39595/ using document root <none> and password file <none>
I20260926 06:23:30.626488 21186 fs_manager.cc:359] Metadata directory not provided
I20260926 06:23:30.626537 21186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:23:30.627813 21186 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:23:30.628453 21201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:23:30.628618 21186 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:23:30.628708 21186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/wal
uuid: "a87d2554f9424d85a88fa4556e64ebfe"
format_stamp: "Formatted at 2026-09-26 06:23:09 on dist-test-slave-pd0t"
server_key: "ac4d2fb15c9faaeca852e9f8848af08d"
server_key_iv: "01a9f6168715205339bbb27fe6a47732"
server_key_version: "encryptionkey@0"
I20260926 06:23:30.629019 21186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:23:30.648475 21186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:23:30.650235 21186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:23:30.650381 21186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:23:30.650619 21186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:23:30.651065 21208 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 06:23:30.652511 21186 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 06:23:30.652597 21186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260926 06:23:30.652643 21186 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 06:23:30.653234 21186 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 06:23:30.653290 21186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260926 06:23:30.653357 21208 tablet_bootstrap.cc:492] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe: Bootstrap starting.
I20260926 06:23:30.654773 21208 log.cc:824] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe: Log is configured to *not* fsync() on all Append() calls
I20260926 06:23:30.656942 21208 tablet_bootstrap.cc:492] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 06:23:30.657227 21208 tablet_bootstrap.cc:492] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe: Bootstrap complete.
I20260926 06:23:30.657387 21208 ts_tablet_manager.cc:1397] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.004s
I20260926 06:23:30.658674 21208 raft_consensus.cc:348] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8900ea32c9e4ca586522252586ec729" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45397 } } peers { permanent_uuid: "515692ec901c470b8b3d3f894cd8bb83" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38333 } } peers { permanent_uuid: "a87d2554f9424d85a88fa4556e64ebfe" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37353 } }
I20260926 06:23:30.658959 21208 raft_consensus.cc:729] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a87d2554f9424d85a88fa4556e64ebfe, State: Initialized, Role: FOLLOWER
I20260926 06:23:30.659132 21208 consensus_queue.cc:260] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8900ea32c9e4ca586522252586ec729" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45397 } } peers { permanent_uuid: "515692ec901c470b8b3d3f894cd8bb83" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38333 } } peers { permanent_uuid: "a87d2554f9424d85a88fa4556e64ebfe" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37353 } }
I20260926 06:23:30.659407 21208 ts_tablet_manager.cc:1428] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 06:23:30.660013 21186 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.131:37353
I20260926 06:23:30.660058 21317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.131:37353 every 8 connection(s)
I20260926 06:23:30.660346 21186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
I20260926 06:23:30.661768 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 21186
Sep 26 06:23:30 dist-test-slave-pd0t krb5kdc[20279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403810, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:23:30.669826 21322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:30.662334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45162 (local address 127.4.202.190:41615)
0926 06:23:30.662431 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0926 06:23:30.662433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:23:30.662442 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:23:30.662499 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:23:30.662500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:23:30.662508 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:23:30.662566 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:23:30.663318 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:30.664085 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:30.665336 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:30.665642 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:30.665717 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:30.666928 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:23:30.666930 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:23:30.666930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:23:30.666937 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:23:30.668643 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:23:30.669018 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:23:30.669018 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:23:30.669019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:23:30.669069 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:23:30.669281 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:23:30.669281 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:23:30.669282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:23:30.669413 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:23:30.669585 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0926 06:23:30.669627 (+ 42us) server_negotiation.cc:299] Negotiation successful
0926 06:23:30.669702 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":59,"threads_started":1}
I20260926 06:23:30.670782 21048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:30.660981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:51165 (local address 127.4.202.190:41615)
0926 06:23:30.661077 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0926 06:23:30.661079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:23:30.661243 (+ 164us) server_negotiation.cc:407] Connection header received
0926 06:23:30.661413 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:23:30.661414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:23:30.661427 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:23:30.661480 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:23:30.662120 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:30.662956 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:30.664372 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:30.664657 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:30.664727 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:30.667600 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:23:30.667602 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:23:30.667602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:23:30.667607 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:23:30.669321 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:23:30.669789 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:23:30.669790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:23:30.669791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:23:30.669850 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:23:30.670168 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:23:30.670168 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:23:30.670169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:23:30.670296 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:23:30.670360 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 06:23:30.670659 (+ 299us) server_negotiation.cc:299] Negotiation successful
0926 06:23:30.670729 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 06:23:30.671030 21320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:30.660934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.131:51165)
0926 06:23:30.661089 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:30.661137 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 06:23:30.661322 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:23:30.661532 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:23:30.661535 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:23:30.661779 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:23:30.661998 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:30.662003 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:30.663090 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:30.663091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:30.664302 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:30.664304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:30.664739 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:30.664740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:30.664827 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:30.665367 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:23:30.665377 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:23:30.667503 (+ 2126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:23:30.669393 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:23:30.669399 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:23:30.669401 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:23:30.669691 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:23:30.669948 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:23:30.669950 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:23:30.669950 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:23:30.670088 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:23:30.670363 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:23:30.670368 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:23:30.670508 (+ 140us) client_negotiation.cc:769] Sending connection context
0926 06:23:30.670642 (+ 134us) client_negotiation.cc:240] Negotiation successful
0926 06:23:30.670770 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
I20260926 06:23:30.671427 21318 heartbeater.cc:347] Connected to a master server at 127.4.202.190:41615
I20260926 06:23:30.671521 21318 heartbeater.cc:464] Registering TS with master...
I20260926 06:23:30.671672 21318 heartbeater.cc:511] Master 127.4.202.190:41615 requested a full tablet report, sending...
I20260926 06:23:30.672137 20857 ts_manager.cc:194] Registered new tserver with Master: a87d2554f9424d85a88fa4556e64ebfe (127.4.202.131:37353)
I20260926 06:23:30.672673 20857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.131@KRBTEST.COM'} at 127.4.202.131:51165
Sep 26 06:23:30 dist-test-slave-pd0t krb5kdc[20279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403810, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for kudu/127.4.202.131@KRBTEST.COM
I20260926 06:23:30.678807 21052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:30.672431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:37353 (local address 127.0.0.1:50514)
0926 06:23:30.672476 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:30.672485 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:23:30.672519 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:23:30.672832 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:23:30.672832 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:23:30.673005 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:23:30.673045 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:30.673046 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:30.673966 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:30.673966 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:30.674422 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:30.674423 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:30.674896 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:30.674897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:30.674957 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:30.675249 (+ 292us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:23:30.675253 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:23:30.676301 (+ 1048us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:23:30.677997 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:23:30.677998 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:23:30.677998 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:23:30.678146 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:23:30.678307 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:23:30.678307 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:23:30.678307 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:23:30.678353 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:23:30.678536 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:23:30.678537 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:23:30.678566 (+ 29us) client_negotiation.cc:769] Sending connection context
0926 06:23:30.678684 (+ 118us) client_negotiation.cc:240] Negotiation successful
0926 06:23:30.678751 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260926 06:23:30.678838 21323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:30.672497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50514 (local address 127.4.202.131:37353)
0926 06:23:30.672656 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0926 06:23:30.672660 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:23:30.672669 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:23:30.672697 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:23:30.672699 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:23:30.672713 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:23:30.672826 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:23:30.673123 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:30.673905 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:30.674488 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:30.674837 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:30.674889 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:30.676374 (+ 1485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:23:30.676379 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:23:30.676380 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:23:30.676385 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:23:30.677923 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:23:30.678205 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:23:30.678207 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:23:30.678207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:23:30.678250 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:23:30.678388 (+ 138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:23:30.678389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:23:30.678389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:23:30.678488 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:23:30.678539 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 06:23:30.678708 (+ 169us) server_negotiation.cc:299] Negotiation successful
0926 06:23:30.678772 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":87,"threads_started":1}
I20260926 06:23:30.679203 21251 tablet_service.cc:1514] Processing DeleteTablet for tablet e3ee6b3a296e48a4a16a125e0be5e049 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:23:10 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:50514
I20260926 06:23:30.679714 21324 tablet_replica.cc:330] stopping tablet replica
I20260926 06:23:30.679813 21324 raft_consensus.cc:2229] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:23:30.679872 21324 raft_consensus.cc:2258] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:23:30.680145 21324 ts_tablet_manager.cc:1905] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:23:30.681169 4906 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:23:30.682497 21324 ts_tablet_manager.cc:1918] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:23:30.682570 21324 log.cc:1196] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/wal/wals/e3ee6b3a296e48a4a16a125e0be5e049
I20260926 06:23:30.682819 21324 ts_tablet_manager.cc:1939] T e3ee6b3a296e48a4a16a125e0be5e049 P a87d2554f9424d85a88fa4556e64ebfe: Deleting consensus metadata
I20260926 06:23:30.683251 20841 catalog_manager.cc:4841] TS a87d2554f9424d85a88fa4556e64ebfe (127.4.202.131:37353): tablet e3ee6b3a296e48a4a16a125e0be5e049 (table test-table [id=cff57303b82d4b63a49890c65188d427]) successfully deleted
I20260926 06:23:30.684010 21048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:30.681589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45166 (local address 127.4.202.190:41615)
0926 06:23:30.681656 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0926 06:23:30.681658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:23:30.681670 (+ 12us) server_negotiation.cc:407] Connection header received
0926 06:23:30.681700 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:23:30.681701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:23:30.681736 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:23:30.681820 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:23:30.681892 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:30.682510 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:30.683466 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:30.683667 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:30.683717 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:30.683850 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:23:30.683881 (+ 31us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0926 06:23:30.683974 (+ 93us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45166: token expired
Metrics: {"server-negotiator.queue_time_us":39}
W20260926 06:23:30.684072 21048 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45166: token expired
W20260926 06:23:30.684217 20789 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.4.202.190:41615)
I20260926 06:23:30.689683 21048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:30.684465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45172 (local address 127.4.202.190:41615)
0926 06:23:30.684503 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 06:23:30.684505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:23:30.684539 (+ 34us) server_negotiation.cc:407] Connection header received
0926 06:23:30.684584 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:23:30.684585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:23:30.684591 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:23:30.684630 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:23:30.685024 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:30.685449 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:30.686165 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:30.686334 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:30.686389 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:30.687529 (+ 1140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:23:30.687531 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:23:30.687531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:23:30.687537 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:23:30.688890 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:23:30.689184 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:23:30.689184 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:23:30.689184 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:23:30.689224 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:23:30.689365 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:23:30.689365 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:23:30.689366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:23:30.689442 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:23:30.689504 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 06:23:30.689580 (+ 76us) server_negotiation.cc:299] Negotiation successful
0926 06:23:30.689628 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:23:30.690788 20857 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45172:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:23:30.692354 20857 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:23:30.694176 21251 tablet_service.cc:1467] Processing CreateTablet for tablet 1ee4a56034c94f5eab503ed58c49df4c (DEFAULT_TABLE table=test-table [id=b7cb22d6a9b24cb696300ac5caeaf37f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:23:30.694178 20979 tablet_service.cc:1467] Processing CreateTablet for tablet 1ee4a56034c94f5eab503ed58c49df4c (DEFAULT_TABLE table=test-table [id=b7cb22d6a9b24cb696300ac5caeaf37f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:23:30.694325 21251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ee4a56034c94f5eab503ed58c49df4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:23:30.694325 20979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ee4a56034c94f5eab503ed58c49df4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:23:30.695729 21208 tablet_bootstrap.cc:492] T 1ee4a56034c94f5eab503ed58c49df4c P a87d2554f9424d85a88fa4556e64ebfe: Bootstrap starting.
I20260926 06:23:30.695849 20937 tablet_bootstrap.cc:492] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729: Bootstrap starting.
I20260926 06:23:30.696148 20937 tablet_bootstrap.cc:654] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729: Neither blocks nor log segments found. Creating new log.
I20260926 06:23:30.696157 21208 tablet_bootstrap.cc:654] T 1ee4a56034c94f5eab503ed58c49df4c P a87d2554f9424d85a88fa4556e64ebfe: Neither blocks nor log segments found. Creating new log.
I20260926 06:23:30.696707 20937 tablet_bootstrap.cc:492] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729: No bootstrap required, opened a new log
I20260926 06:23:30.696785 20937 ts_tablet_manager.cc:1397] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260926 06:23:30.697113 20937 raft_consensus.cc:348] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515692ec901c470b8b3d3f894cd8bb83" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38333 } } peers { permanent_uuid: "c8900ea32c9e4ca586522252586ec729" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45397 } } peers { permanent_uuid: "a87d2554f9424d85a88fa4556e64ebfe" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37353 } }
I20260926 06:23:30.697202 20937 raft_consensus.cc:374] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:23:30.697254 20937 raft_consensus.cc:729] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8900ea32c9e4ca586522252586ec729, State: Initialized, Role: FOLLOWER
I20260926 06:23:30.697306 20937 consensus_queue.cc:260] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515692ec901c470b8b3d3f894cd8bb83" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38333 } } peers { permanent_uuid: "c8900ea32c9e4ca586522252586ec729" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45397 } } peers { permanent_uuid: "a87d2554f9424d85a88fa4556e64ebfe" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37353 } }
I20260926 06:23:30.697409 20937 ts_tablet_manager.cc:1428] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 06:23:30.697592 21208 tablet_bootstrap.cc:492] T 1ee4a56034c94f5eab503ed58c49df4c P a87d2554f9424d85a88fa4556e64ebfe: No bootstrap required, opened a new log
I20260926 06:23:30.697630 21208 ts_tablet_manager.cc:1397] T 1ee4a56034c94f5eab503ed58c49df4c P a87d2554f9424d85a88fa4556e64ebfe: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 06:23:30.697785 21052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:30.693995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:38333 (local address 127.0.0.1:34242)
0926 06:23:30.694063 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:30.694073 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:23:30.694111 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:23:30.694462 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:23:30.694462 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:23:30.694464 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:23:30.694506 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:30.694507 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:30.695658 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:30.695659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:30.696612 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:30.696615 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:30.697548 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:30.697549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:30.697625 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:30.697644 (+ 19us) client_negotiation.cc:769] Sending connection context
0926 06:23:30.697695 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 06:23:30.697730 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260926 06:23:30.697906 21208 raft_consensus.cc:348] T 1ee4a56034c94f5eab503ed58c49df4c P a87d2554f9424d85a88fa4556e64ebfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515692ec901c470b8b3d3f894cd8bb83" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38333 } } peers { permanent_uuid: "c8900ea32c9e4ca586522252586ec729" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45397 } } peers { permanent_uuid: "a87d2554f9424d85a88fa4556e64ebfe" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37353 } }
I20260926 06:23:30.697975 21208 raft_consensus.cc:374] T 1ee4a56034c94f5eab503ed58c49df4c P a87d2554f9424d85a88fa4556e64ebfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:23:30.698020 21326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:30.694121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34242 (local address 127.4.202.130:38333)
0926 06:23:30.694282 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 06:23:30.694286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:23:30.694296 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:23:30.694335 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:23:30.694337 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:23:30.694352 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:23:30.694564 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:23:30.694717 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:30.695569 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:30.696706 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:30.697391 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:30.697459 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:30.697513 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 06:23:30.697890 (+ 377us) server_negotiation.cc:299] Negotiation successful
0926 06:23:30.697934 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":92,"threads_started":1}
I20260926 06:23:30.698046 21208 raft_consensus.cc:729] T 1ee4a56034c94f5eab503ed58c49df4c P a87d2554f9424d85a88fa4556e64ebfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a87d2554f9424d85a88fa4556e64ebfe, State: Initialized, Role: FOLLOWER
I20260926 06:23:30.698100 21208 consensus_queue.cc:260] T 1ee4a56034c94f5eab503ed58c49df4c P a87d2554f9424d85a88fa4556e64ebfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515692ec901c470b8b3d3f894cd8bb83" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38333 } } peers { permanent_uuid: "c8900ea32c9e4ca586522252586ec729" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45397 } } peers { permanent_uuid: "a87d2554f9424d85a88fa4556e64ebfe" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37353 } }
I20260926 06:23:30.698220 21208 ts_tablet_manager.cc:1428] T 1ee4a56034c94f5eab503ed58c49df4c P a87d2554f9424d85a88fa4556e64ebfe: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 06:23:30.698258 21318 heartbeater.cc:503] Master 127.4.202.190:41615 was elected leader, sending a full tablet report...
I20260926 06:23:30.698807 21117 tablet_service.cc:1467] Processing CreateTablet for tablet 1ee4a56034c94f5eab503ed58c49df4c (DEFAULT_TABLE table=test-table [id=b7cb22d6a9b24cb696300ac5caeaf37f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:23:30.699203 21117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ee4a56034c94f5eab503ed58c49df4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:23:30.701844 21329 tablet_bootstrap.cc:492] T 1ee4a56034c94f5eab503ed58c49df4c P 515692ec901c470b8b3d3f894cd8bb83: Bootstrap starting.
I20260926 06:23:30.702491 21329 tablet_bootstrap.cc:654] T 1ee4a56034c94f5eab503ed58c49df4c P 515692ec901c470b8b3d3f894cd8bb83: Neither blocks nor log segments found. Creating new log.
I20260926 06:23:30.703008 21329 log.cc:824] T 1ee4a56034c94f5eab503ed58c49df4c P 515692ec901c470b8b3d3f894cd8bb83: Log is configured to *not* fsync() on all Append() calls
I20260926 06:23:30.703718 21329 tablet_bootstrap.cc:492] T 1ee4a56034c94f5eab503ed58c49df4c P 515692ec901c470b8b3d3f894cd8bb83: No bootstrap required, opened a new log
I20260926 06:23:30.703815 21329 ts_tablet_manager.cc:1397] T 1ee4a56034c94f5eab503ed58c49df4c P 515692ec901c470b8b3d3f894cd8bb83: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:23:30.705173 21329 raft_consensus.cc:348] T 1ee4a56034c94f5eab503ed58c49df4c P 515692ec901c470b8b3d3f894cd8bb83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515692ec901c470b8b3d3f894cd8bb83" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38333 } } peers { permanent_uuid: "c8900ea32c9e4ca586522252586ec729" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45397 } } peers { permanent_uuid: "a87d2554f9424d85a88fa4556e64ebfe" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37353 } }
I20260926 06:23:30.705333 21329 raft_consensus.cc:374] T 1ee4a56034c94f5eab503ed58c49df4c P 515692ec901c470b8b3d3f894cd8bb83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:23:30.705379 21329 raft_consensus.cc:729] T 1ee4a56034c94f5eab503ed58c49df4c P 515692ec901c470b8b3d3f894cd8bb83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 515692ec901c470b8b3d3f894cd8bb83, State: Initialized, Role: FOLLOWER
I20260926 06:23:30.705502 21329 consensus_queue.cc:260] T 1ee4a56034c94f5eab503ed58c49df4c P 515692ec901c470b8b3d3f894cd8bb83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515692ec901c470b8b3d3f894cd8bb83" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38333 } } peers { permanent_uuid: "c8900ea32c9e4ca586522252586ec729" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45397 } } peers { permanent_uuid: "a87d2554f9424d85a88fa4556e64ebfe" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37353 } }
I20260926 06:23:30.705771 21329 ts_tablet_manager.cc:1428] T 1ee4a56034c94f5eab503ed58c49df4c P 515692ec901c470b8b3d3f894cd8bb83: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:23:30.705785 21183 heartbeater.cc:503] Master 127.4.202.190:41615 was elected leader, sending a full tablet report...
W20260926 06:23:30.771775 21184 tablet.cc:2369] T 1ee4a56034c94f5eab503ed58c49df4c P 515692ec901c470b8b3d3f894cd8bb83: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 06:23:30.885473 21046 tablet.cc:2369] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 06:23:30.910935 21319 tablet.cc:2369] T 1ee4a56034c94f5eab503ed58c49df4c P a87d2554f9424d85a88fa4556e64ebfe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:23:31.023125 21308 raft_consensus.cc:482] T 1ee4a56034c94f5eab503ed58c49df4c P a87d2554f9424d85a88fa4556e64ebfe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:23:31.023272 21308 raft_consensus.cc:504] T 1ee4a56034c94f5eab503ed58c49df4c P a87d2554f9424d85a88fa4556e64ebfe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515692ec901c470b8b3d3f894cd8bb83" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38333 } } peers { permanent_uuid: "c8900ea32c9e4ca586522252586ec729" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45397 } } peers { permanent_uuid: "a87d2554f9424d85a88fa4556e64ebfe" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37353 } }
I20260926 06:23:31.023618 21308 leader_election.cc:288] T 1ee4a56034c94f5eab503ed58c49df4c P a87d2554f9424d85a88fa4556e64ebfe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 515692ec901c470b8b3d3f894cd8bb83 (127.4.202.130:38333), c8900ea32c9e4ca586522252586ec729 (127.4.202.129:45397)
I20260926 06:23:31.027640 21332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:31.023872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:50027 (local address 127.4.202.129:45397)
0926 06:23:31.024011 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 06:23:31.024013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:23:31.024023 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:23:31.024055 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:23:31.024055 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:23:31.024077 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:23:31.024217 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:23:31.024460 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:31.025207 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:31.026688 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:31.027293 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:31.027362 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:31.027413 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 06:23:31.027514 (+ 101us) server_negotiation.cc:299] Negotiation successful
0926 06:23:31.027554 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":71,"threads_started":1}
I20260926 06:23:31.027639 21326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:31.023849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:51349 (local address 127.4.202.130:38333)
0926 06:23:31.023891 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 06:23:31.023893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:23:31.024131 (+ 238us) server_negotiation.cc:407] Connection header received
0926 06:23:31.024158 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:23:31.024159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:23:31.024169 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:23:31.024222 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:23:31.024460 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:31.025182 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:31.026688 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:31.027212 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:31.027290 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:31.027300 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 06:23:31.027514 (+ 214us) server_negotiation.cc:299] Negotiation successful
0926 06:23:31.027553 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260926 06:23:31.027650 21320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:31.023802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:45397 (local address 127.4.202.131:50027)
0926 06:23:31.023866 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:31.023879 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 06:23:31.023948 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:23:31.024244 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:23:31.024245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:23:31.024246 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:23:31.024347 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:31.024349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:31.025282 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:31.025283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:31.026571 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:31.026574 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:31.027366 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:31.027366 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:31.027429 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:31.027439 (+ 10us) client_negotiation.cc:769] Sending connection context
0926 06:23:31.027498 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 06:23:31.027544 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":113}
I20260926 06:23:31.027680 21333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:31.023804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:38333 (local address 127.4.202.131:51349)
0926 06:23:31.024059 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:31.024067 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 06:23:31.024111 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:23:31.024244 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:23:31.024245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:23:31.024246 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:23:31.024347 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:31.024349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:31.025280 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:31.025281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:31.026571 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:31.026574 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:31.027303 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:31.027304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:31.027409 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:31.027434 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 06:23:31.027498 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 06:23:31.027541 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":116,"thread_start_us":112,"threads_started":1}
I20260926 06:23:31.028090 21137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ee4a56034c94f5eab503ed58c49df4c" candidate_uuid: "a87d2554f9424d85a88fa4556e64ebfe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "515692ec901c470b8b3d3f894cd8bb83" is_pre_election: true
I20260926 06:23:31.028110 20999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ee4a56034c94f5eab503ed58c49df4c" candidate_uuid: "a87d2554f9424d85a88fa4556e64ebfe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8900ea32c9e4ca586522252586ec729" is_pre_election: true
I20260926 06:23:31.028254 21137 raft_consensus.cc:2454] T 1ee4a56034c94f5eab503ed58c49df4c P 515692ec901c470b8b3d3f894cd8bb83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a87d2554f9424d85a88fa4556e64ebfe in term 0.
I20260926 06:23:31.028254 20999 raft_consensus.cc:2454] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a87d2554f9424d85a88fa4556e64ebfe in term 0.
I20260926 06:23:31.028476 21204 leader_election.cc:302] T 1ee4a56034c94f5eab503ed58c49df4c P a87d2554f9424d85a88fa4556e64ebfe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a87d2554f9424d85a88fa4556e64ebfe, c8900ea32c9e4ca586522252586ec729; no voters:
I20260926 06:23:31.028644 21308 raft_consensus.cc:2781] T 1ee4a56034c94f5eab503ed58c49df4c P a87d2554f9424d85a88fa4556e64ebfe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:23:31.028707 21308 raft_consensus.cc:482] T 1ee4a56034c94f5eab503ed58c49df4c P a87d2554f9424d85a88fa4556e64ebfe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:23:31.028721 21308 raft_consensus.cc:3037] T 1ee4a56034c94f5eab503ed58c49df4c P a87d2554f9424d85a88fa4556e64ebfe [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:23:31.029434 21308 raft_consensus.cc:504] T 1ee4a56034c94f5eab503ed58c49df4c P a87d2554f9424d85a88fa4556e64ebfe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515692ec901c470b8b3d3f894cd8bb83" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38333 } } peers { permanent_uuid: "c8900ea32c9e4ca586522252586ec729" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45397 } } peers { permanent_uuid: "a87d2554f9424d85a88fa4556e64ebfe" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37353 } }
I20260926 06:23:31.029570 21308 leader_election.cc:288] T 1ee4a56034c94f5eab503ed58c49df4c P a87d2554f9424d85a88fa4556e64ebfe [CANDIDATE]: Term 1 election: Requested vote from peers 515692ec901c470b8b3d3f894cd8bb83 (127.4.202.130:38333), c8900ea32c9e4ca586522252586ec729 (127.4.202.129:45397)
I20260926 06:23:31.029716 20999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ee4a56034c94f5eab503ed58c49df4c" candidate_uuid: "a87d2554f9424d85a88fa4556e64ebfe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8900ea32c9e4ca586522252586ec729"
I20260926 06:23:31.029752 21137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ee4a56034c94f5eab503ed58c49df4c" candidate_uuid: "a87d2554f9424d85a88fa4556e64ebfe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "515692ec901c470b8b3d3f894cd8bb83"
I20260926 06:23:31.029805 20999 raft_consensus.cc:3037] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:23:31.029812 21137 raft_consensus.cc:3037] T 1ee4a56034c94f5eab503ed58c49df4c P 515692ec901c470b8b3d3f894cd8bb83 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:23:31.030750 21137 raft_consensus.cc:2454] T 1ee4a56034c94f5eab503ed58c49df4c P 515692ec901c470b8b3d3f894cd8bb83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a87d2554f9424d85a88fa4556e64ebfe in term 1.
I20260926 06:23:31.030787 20999 raft_consensus.cc:2454] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a87d2554f9424d85a88fa4556e64ebfe in term 1.
I20260926 06:23:31.030933 21203 leader_election.cc:302] T 1ee4a56034c94f5eab503ed58c49df4c P a87d2554f9424d85a88fa4556e64ebfe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 515692ec901c470b8b3d3f894cd8bb83, a87d2554f9424d85a88fa4556e64ebfe; no voters:
I20260926 06:23:31.031082 21308 raft_consensus.cc:2781] T 1ee4a56034c94f5eab503ed58c49df4c P a87d2554f9424d85a88fa4556e64ebfe [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:23:31.031210 21308 raft_consensus.cc:686] T 1ee4a56034c94f5eab503ed58c49df4c P a87d2554f9424d85a88fa4556e64ebfe [term 1 LEADER]: Becoming Leader. State: Replica: a87d2554f9424d85a88fa4556e64ebfe, State: Running, Role: LEADER
I20260926 06:23:31.031287 21308 consensus_queue.cc:237] T 1ee4a56034c94f5eab503ed58c49df4c P a87d2554f9424d85a88fa4556e64ebfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515692ec901c470b8b3d3f894cd8bb83" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38333 } } peers { permanent_uuid: "c8900ea32c9e4ca586522252586ec729" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45397 } } peers { permanent_uuid: "a87d2554f9424d85a88fa4556e64ebfe" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37353 } }
I20260926 06:23:31.032035 20856 catalog_manager.cc:5495] T 1ee4a56034c94f5eab503ed58c49df4c P a87d2554f9424d85a88fa4556e64ebfe reported cstate change: term changed from 0 to 1, leader changed from <none> to a87d2554f9424d85a88fa4556e64ebfe (127.4.202.131). New cstate: current_term: 1 leader_uuid: "a87d2554f9424d85a88fa4556e64ebfe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515692ec901c470b8b3d3f894cd8bb83" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8900ea32c9e4ca586522252586ec729" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a87d2554f9424d85a88fa4556e64ebfe" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37353 } health_report { overall_health: HEALTHY } } }
I20260926 06:23:31.049301 21323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:31.047213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50516 (local address 127.4.202.131:37353)
0926 06:23:31.047251 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 06:23:31.047253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:23:31.047283 (+ 30us) server_negotiation.cc:407] Connection header received
0926 06:23:31.047343 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:23:31.047343 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:23:31.047358 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:23:31.047415 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:23:31.047506 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:31.047955 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:31.048633 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:31.048826 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:31.048882 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:31.048997 (+ 115us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:23:31.049080 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:23:31.049139 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 06:23:31.049176 (+ 37us) server_negotiation.cc:299] Negotiation successful
0926 06:23:31.049192 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 06:23:31.050724 21137 raft_consensus.cc:1261] T 1ee4a56034c94f5eab503ed58c49df4c P 515692ec901c470b8b3d3f894cd8bb83 [term 1 FOLLOWER]: Refusing update from remote peer a87d2554f9424d85a88fa4556e64ebfe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:23:31.050724 20999 raft_consensus.cc:1261] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [term 1 FOLLOWER]: Refusing update from remote peer a87d2554f9424d85a88fa4556e64ebfe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:23:31.051002 21308 consensus_queue.cc:1035] T 1ee4a56034c94f5eab503ed58c49df4c P a87d2554f9424d85a88fa4556e64ebfe [LEADER]: Connected to new peer: Peer: permanent_uuid: "515692ec901c470b8b3d3f894cd8bb83" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:23:31.051165 21334 consensus_queue.cc:1035] T 1ee4a56034c94f5eab503ed58c49df4c P a87d2554f9424d85a88fa4556e64ebfe [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8900ea32c9e4ca586522252586ec729" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:23:31.053987 21338 mvcc.cc:205] Tried to move back new op lower bound from 7333494010061254656 to 7333494009984663552. Current Snapshot: MvccSnapshot[applied={T|T < 7333494010061254656}]
I20260926 06:23:31.055050 20857 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:23:31.055178 20857 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:23:31.055812 20857 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099: Sending DeleteTablet for 3 replicas of tablet 1ee4a56034c94f5eab503ed58c49df4c
I20260926 06:23:31.056070 20979 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ee4a56034c94f5eab503ed58c49df4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:23:31 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:55116
I20260926 06:23:31.056131 21251 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ee4a56034c94f5eab503ed58c49df4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:23:31 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:50514
I20260926 06:23:31.056128 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 20895
I20260926 06:23:31.056229 21324 tablet_replica.cc:330] T 1ee4a56034c94f5eab503ed58c49df4c P a87d2554f9424d85a88fa4556e64ebfe: stopping tablet replica
I20260926 06:23:31.056289 21324 raft_consensus.cc:2229] T 1ee4a56034c94f5eab503ed58c49df4c P a87d2554f9424d85a88fa4556e64ebfe [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:23:31.056361 21324 raft_consensus.cc:2258] T 1ee4a56034c94f5eab503ed58c49df4c P a87d2554f9424d85a88fa4556e64ebfe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:23:31.056705 21117 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ee4a56034c94f5eab503ed58c49df4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:23:31 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:34242
I20260926 06:23:31.056792 21324 ts_tablet_manager.cc:1905] T 1ee4a56034c94f5eab503ed58c49df4c P a87d2554f9424d85a88fa4556e64ebfe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:23:31.056911 21347 tablet_replica.cc:330] T 1ee4a56034c94f5eab503ed58c49df4c P 515692ec901c470b8b3d3f894cd8bb83: stopping tablet replica
I20260926 06:23:31.057010 21347 raft_consensus.cc:2229] T 1ee4a56034c94f5eab503ed58c49df4c P 515692ec901c470b8b3d3f894cd8bb83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:23:31.057158 21347 raft_consensus.cc:2258] T 1ee4a56034c94f5eab503ed58c49df4c P 515692ec901c470b8b3d3f894cd8bb83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:23:31.057621 21347 ts_tablet_manager.cc:1905] T 1ee4a56034c94f5eab503ed58c49df4c P 515692ec901c470b8b3d3f894cd8bb83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:23:31.058815 21324 ts_tablet_manager.cc:1918] T 1ee4a56034c94f5eab503ed58c49df4c P a87d2554f9424d85a88fa4556e64ebfe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:23:31.058877 21324 log.cc:1196] T 1ee4a56034c94f5eab503ed58c49df4c P a87d2554f9424d85a88fa4556e64ebfe: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/wal/wals/1ee4a56034c94f5eab503ed58c49df4c
I20260926 06:23:31.059098 21324 ts_tablet_manager.cc:1939] T 1ee4a56034c94f5eab503ed58c49df4c P a87d2554f9424d85a88fa4556e64ebfe: Deleting consensus metadata
I20260926 06:23:31.059316 21347 ts_tablet_manager.cc:1918] T 1ee4a56034c94f5eab503ed58c49df4c P 515692ec901c470b8b3d3f894cd8bb83: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:23:31.059425 21347 log.cc:1196] T 1ee4a56034c94f5eab503ed58c49df4c P 515692ec901c470b8b3d3f894cd8bb83: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/wal/wals/1ee4a56034c94f5eab503ed58c49df4c
I20260926 06:23:31.059477 20841 catalog_manager.cc:4841] TS a87d2554f9424d85a88fa4556e64ebfe (127.4.202.131:37353): tablet 1ee4a56034c94f5eab503ed58c49df4c (table test-table [id=b7cb22d6a9b24cb696300ac5caeaf37f]) successfully deleted
I20260926 06:23:31.059710 21347 ts_tablet_manager.cc:1939] T 1ee4a56034c94f5eab503ed58c49df4c P 515692ec901c470b8b3d3f894cd8bb83: Deleting consensus metadata
I20260926 06:23:31.060113 20842 catalog_manager.cc:4841] TS 515692ec901c470b8b3d3f894cd8bb83 (127.4.202.130:38333): tablet 1ee4a56034c94f5eab503ed58c49df4c (table test-table [id=b7cb22d6a9b24cb696300ac5caeaf37f]) successfully deleted
W20260926 06:23:31.062939 20843 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.202.129:45397 (error 108)
I20260926 06:23:31.063212 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 21050
I20260926 06:23:31.063791 21052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:31.063316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:45397 (local address 127.0.0.1:55124)
0926 06:23:31.063691 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:31.063750 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.129:45397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358}
W20260926 06:23:31.063970 20843 catalog_manager.cc:4584] TS c8900ea32c9e4ca586522252586ec729 (127.4.202.129:45397): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1ee4a56034c94f5eab503ed58c49df4c: Network error: Client connection negotiation failed: client connection to 127.4.202.129:45397: connect: Connection refused (error 111)
I20260926 06:23:31.068662 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 21186
I20260926 06:23:31.073566 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 20824
I20260926 06:23:51.078610 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:41615
--webserver_interface=127.4.202.190
--webserver_port=46051
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:41921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:41615
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:23:51.157967 21349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:23:51.158196 21349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:23:51.158229 21349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:23:51.159409 21349 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 06:23:51.159469 21349 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 06:23:51.159492 21349 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 06:23:51.159510 21349 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:23:51.159538 21349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:23:51.159571 21349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:23:51.159595 21349 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:23:51.159621 21349 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:23:51.161001 21349 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:41921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:41615
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:41615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=46051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:23:51.161263 21349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:23:51.161477 21349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:23:51.166330 21357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:23:51.166299 21354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:23:51.166477 21349 server_base.cc:1034] running on GCE node
W20260926 06:23:51.166301 21355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:23:51.166849 21349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:23:51.167171 21349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:23:51.168349 21349 hybrid_clock.cc:648] HybridClock initialized: now 1790403831168328 us; error 37 us; skew 500 ppm
Sep 26 06:23:51 dist-test-slave-pd0t krb5kdc[20279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403831, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:23:51.170799 21349 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:23:51.171237 21349 webserver.cc:460] Webserver started at http://127.4.202.190:46051/ using document root <none> and password file <none>
I20260926 06:23:51.171474 21349 fs_manager.cc:359] Metadata directory not provided
I20260926 06:23:51.171543 21349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:23:51.172837 21349 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:23:51.173545 21364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:23:51.173727 21349 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:23:51.173846 21349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "90b703134eac456597feb5ca43537099"
format_stamp: "Formatted at 2026-09-26 06:23:09 on dist-test-slave-pd0t"
server_key: "931169c4c3d8c12ba5b4407b7abb25a4"
server_key_iv: "3d8f79484670ed072ea8083ad0646833"
server_key_version: "encryptionkey@0"
I20260926 06:23:51.174211 21349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:23:51.197343 21349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:23:51.199172 21349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:23:51.199332 21349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:23:51.203016 21349 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:41615
I20260926 06:23:51.203063 21416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:41615 every 8 connection(s)
I20260926 06:23:51.203430 21349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:23:51.204770 21417 sys_catalog.cc:263] Verifying existing consensus state
I20260926 06:23:51.205391 21417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099: Bootstrap starting.
I20260926 06:23:51.213377 21417 log.cc:824] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099: Log is configured to *not* fsync() on all Append() calls
I20260926 06:23:51.213508 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 21349
I20260926 06:23:51.213819 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.129:45397
--local_ip_for_outbound_sockets=127.4.202.129
--tserver_master_addrs=127.4.202.190:41615
--webserver_port=35235
--webserver_interface=127.4.202.129
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:41921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:23:51.215994 21417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 06:23:51.216336 21417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099: Bootstrap complete.
I20260926 06:23:51.218516 21417 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90b703134eac456597feb5ca43537099" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 41615 } }
I20260926 06:23:51.218794 21417 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90b703134eac456597feb5ca43537099, State: Initialized, Role: FOLLOWER
I20260926 06:23:51.218954 21417 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90b703134eac456597feb5ca43537099" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 41615 } }
I20260926 06:23:51.219012 21417 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:23:51.219056 21417 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:23:51.219110 21417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [term 2 FOLLOWER]: Advancing to term 3
I20260926 06:23:51.220037 21417 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90b703134eac456597feb5ca43537099" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 41615 } }
I20260926 06:23:51.220129 21417 leader_election.cc:302] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90b703134eac456597feb5ca43537099; no voters:
I20260926 06:23:51.220295 21417 leader_election.cc:288] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260926 06:23:51.220359 21421 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [term 3 FOLLOWER]: Leader election won for term 3
I20260926 06:23:51.220534 21421 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [term 3 LEADER]: Becoming Leader. State: Replica: 90b703134eac456597feb5ca43537099, State: Running, Role: LEADER
I20260926 06:23:51.220574 21417 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:23:51.220721 21421 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90b703134eac456597feb5ca43537099" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 41615 } }
I20260926 06:23:51.221207 21423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90b703134eac456597feb5ca43537099. Latest consensus state: current_term: 3 leader_uuid: "90b703134eac456597feb5ca43537099" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90b703134eac456597feb5ca43537099" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 41615 } } }
I20260926 06:23:51.221216 21422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "90b703134eac456597feb5ca43537099" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90b703134eac456597feb5ca43537099" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 41615 } } }
I20260926 06:23:51.221393 21423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [sys.catalog]: This master's current role is: LEADER
I20260926 06:23:51.221405 21422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099 [sys.catalog]: This master's current role is: LEADER
I20260926 06:23:51.221648 21430 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:23:51.222630 21430 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:23:51.222795 21430 catalog_manager.cc:1259] Loaded cluster ID: 286568cd652242be8b3d784b16bff7a8
I20260926 06:23:51.223037 21430 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:23:51.224215 21430 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:23:51.224385 21430 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099: Loaded TSK: 0
I20260926 06:23:51.224433 21430 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 90b703134eac456597feb5ca43537099: Loaded TSK: 1
F20260926 06:23:51.224658 21430 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f22550d19ad google::LogMessage::Fail() at ??:0
@ 0x7f22550d488a google::LogMessage::SendToLog() at ??:0
@ 0x7f22550d14de google::LogMessage::Flush() at ??:0
@ 0x7f22550d2a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f225783e104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f225783bd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f225d02b1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f225d02b48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f225d02b594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f225cff1c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f225cffbad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f2256f9f0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f2256f94954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f2257df86db start_thread at ??:0
@ 0x7f2253bcf61f clone at ??:0
W20260926 06:23:51.328575 21420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:23:51.328809 21420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:23:51.328845 21420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:23:51.329996 21420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:23:51.330065 21420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:23:51.330111 21420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.129
I20260926 06:23:51.331633 21420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:41921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.129:45397
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.202.129
--webserver_port=35235
--tserver_master_addrs=127.4.202.190:41615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.129
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:23:51.331887 21420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:23:51.332116 21420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:23:51.337055 21461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:23:51.337085 21459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:23:51.337086 21458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:23:51.337810 21420 server_base.cc:1034] running on GCE node
I20260926 06:23:51.338032 21420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:23:51.338312 21420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:23:51.339464 21420 hybrid_clock.cc:648] HybridClock initialized: now 1790403831339454 us; error 26 us; skew 500 ppm
Sep 26 06:23:51 dist-test-slave-pd0t krb5kdc[20279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403831, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:23:51.341873 21420 init.cc:377] Logged in from keytab as kudu/127.4.202.129@KRBTEST.COM (short username kudu)
I20260926 06:23:51.342366 21420 webserver.cc:460] Webserver started at http://127.4.202.129:35235/ using document root <none> and password file <none>
I20260926 06:23:51.342664 21420 fs_manager.cc:359] Metadata directory not provided
I20260926 06:23:51.342710 21420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:23:51.344027 21420 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:23:51.344720 21468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:23:51.344900 21420 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:23:51.344962 21420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/wal
uuid: "c8900ea32c9e4ca586522252586ec729"
format_stamp: "Formatted at 2026-09-26 06:23:09 on dist-test-slave-pd0t"
server_key: "25409706138cc6fc0dddab57397c20fa"
server_key_iv: "b5ea3da294d48df3e8520d03197b02ce"
server_key_version: "encryptionkey@0"
I20260926 06:23:51.345299 21420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:23:51.352409 21420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:23:51.354691 21420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:23:51.354849 21420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:23:51.355093 21420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:23:51.355557 21475 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 06:23:51.356945 21420 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 06:23:51.356995 21420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260926 06:23:51.357024 21420 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 06:23:51.357633 21420 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 06:23:51.357674 21420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260926 06:23:51.357724 21475 tablet_bootstrap.cc:492] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729: Bootstrap starting.
I20260926 06:23:51.365383 21420 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.129:45397
I20260926 06:23:51.365525 21582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.129:45397 every 8 connection(s)
I20260926 06:23:51.366040 21420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
I20260926 06:23:51.369037 21475 log.cc:824] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729: Log is configured to *not* fsync() on all Append() calls
I20260926 06:23:51.369194 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 21420
I20260926 06:23:51.369513 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.130:38333
--local_ip_for_outbound_sockets=127.4.202.130
--tserver_master_addrs=127.4.202.190:41615
--webserver_port=41613
--webserver_interface=127.4.202.130
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:41921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:23:51.371477 21475 tablet_bootstrap.cc:492] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 06:23:51.371798 21475 tablet_bootstrap.cc:492] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729: Bootstrap complete.
I20260926 06:23:51.372977 21475 ts_tablet_manager.cc:1397] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.009s
I20260926 06:23:51.374792 21475 raft_consensus.cc:348] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515692ec901c470b8b3d3f894cd8bb83" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38333 } } peers { permanent_uuid: "c8900ea32c9e4ca586522252586ec729" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45397 } } peers { permanent_uuid: "a87d2554f9424d85a88fa4556e64ebfe" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37353 } }
I20260926 06:23:51.375211 21475 raft_consensus.cc:729] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8900ea32c9e4ca586522252586ec729, State: Initialized, Role: FOLLOWER
I20260926 06:23:51.375398 21475 consensus_queue.cc:260] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515692ec901c470b8b3d3f894cd8bb83" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38333 } } peers { permanent_uuid: "c8900ea32c9e4ca586522252586ec729" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45397 } } peers { permanent_uuid: "a87d2554f9424d85a88fa4556e64ebfe" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37353 } }
I20260926 06:23:51.375692 21475 ts_tablet_manager.cc:1428] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
W20260926 06:23:51.429909 21585 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:23:51.366773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.129:35847)
0926 06:23:51.366993 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:51.367043 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 06:23:51.367834 (+ 791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:23:51.429749 (+ 61915us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":91,"threads_started":1}
W20260926 06:23:51.430209 21583 heartbeater.cc:644] Failed to heartbeat to 127.4.202.190:41615 (0 consecutive failures): Network error: Failed to ping master at 127.4.202.190:41615: Client connection negotiation failed: client connection to 127.4.202.190:41615: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 06:23:51.430629 21585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:51.430464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.129:41277)
0926 06:23:51.430519 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:51.430561 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 06:23:51.430980 21585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:51.430886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.129:60975)
0926 06:23:51.430935 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:51.430959 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260926 06:23:51.431115 21583 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 06:23:51.453363 21587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:23:51.453553 21587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:23:51.453577 21587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:23:51.454746 21587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:23:51.454787 21587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:23:51.454813 21587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.130
I20260926 06:23:51.456190 21587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:41921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.130:38333
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.202.130
--webserver_port=41613
--tserver_master_addrs=127.4.202.190:41615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.130
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:23:51.456362 21587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:23:51.456518 21587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:23:51.461442 21593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:23:51.461467 21594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:23:51.461493 21587 server_base.cc:1034] running on GCE node
W20260926 06:23:51.461464 21596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:23:51.461792 21587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:23:51.462025 21587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:23:51.463193 21587 hybrid_clock.cc:648] HybridClock initialized: now 1790403831463192 us; error 36 us; skew 500 ppm
Sep 26 06:23:51 dist-test-slave-pd0t krb5kdc[20279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403831, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:23:51.465806 21587 init.cc:377] Logged in from keytab as kudu/127.4.202.130@KRBTEST.COM (short username kudu)
I20260926 06:23:51.466214 21587 webserver.cc:460] Webserver started at http://127.4.202.130:41613/ using document root <none> and password file <none>
I20260926 06:23:51.466437 21587 fs_manager.cc:359] Metadata directory not provided
I20260926 06:23:51.466516 21587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:23:51.467898 21587 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:23:51.468524 21603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:23:51.468828 21587 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:23:51.468937 21587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/wal
uuid: "515692ec901c470b8b3d3f894cd8bb83"
format_stamp: "Formatted at 2026-09-26 06:23:09 on dist-test-slave-pd0t"
server_key: "30cc733beab913f1436d6311c50c8000"
server_key_iv: "8c8a47d736497e90060cd9acd6d59f6c"
server_key_version: "encryptionkey@0"
I20260926 06:23:51.469305 21587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:23:51.482036 21587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:23:51.484091 21587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:23:51.484251 21587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:23:51.484467 21587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:23:51.484766 21587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:23:51.484826 21587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:23:51.484859 21587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:23:51.484885 21587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:23:51.490813 21587 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.130:38333
I20260926 06:23:51.490860 21716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.130:38333 every 8 connection(s)
I20260926 06:23:51.491217 21587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
I20260926 06:23:51.492295 21719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:51.491917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.130:52445)
0926 06:23:51.492083 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:51.492155 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":79,"threads_started":1}
W20260926 06:23:51.492715 21717 heartbeater.cc:644] Failed to heartbeat to 127.4.202.190:41615 (0 consecutive failures): Network error: Failed to ping master at 127.4.202.190:41615: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
I20260926 06:23:51.492990 21719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:51.492893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.130:47383)
0926 06:23:51.492939 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:51.492969 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260926 06:23:51.493343 21719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:51.493264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.130:54741)
0926 06:23:51.493306 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:51.493329 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260926 06:23:51.493520 21717 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 06:23:51.495280 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 21587
I20260926 06:23:51.495522 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.131:37353
--local_ip_for_outbound_sockets=127.4.202.131
--tserver_master_addrs=127.4.202.190:41615
--webserver_port=39595
--webserver_interface=127.4.202.131
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:41921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:23:51.574646 21720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:23:51.574862 21720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:23:51.574899 21720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:23:51.576021 21720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:23:51.576087 21720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:23:51.576128 21720 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.131
I20260926 06:23:51.577561 21720 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:41921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.131:37353
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.202.131
--webserver_port=39595
--tserver_master_addrs=127.4.202.190:41615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.131
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:23:51.577819 21720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:23:51.578037 21720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:23:51.582839 21726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:23:51.583046 21720 server_base.cc:1034] running on GCE node
W20260926 06:23:51.582834 21725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:23:51.582939 21728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:23:51.583362 21720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:23:51.583617 21720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:23:51.584776 21720 hybrid_clock.cc:648] HybridClock initialized: now 1790403831584760 us; error 29 us; skew 500 ppm
Sep 26 06:23:51 dist-test-slave-pd0t krb5kdc[20279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403831, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:23:51.587181 21720 init.cc:377] Logged in from keytab as kudu/127.4.202.131@KRBTEST.COM (short username kudu)
I20260926 06:23:51.587635 21720 webserver.cc:460] Webserver started at http://127.4.202.131:39595/ using document root <none> and password file <none>
I20260926 06:23:51.587859 21720 fs_manager.cc:359] Metadata directory not provided
I20260926 06:23:51.587927 21720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:23:51.589198 21720 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:23:51.589839 21735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:23:51.589998 21720 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 06:23:51.590101 21720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/wal
uuid: "a87d2554f9424d85a88fa4556e64ebfe"
format_stamp: "Formatted at 2026-09-26 06:23:09 on dist-test-slave-pd0t"
server_key: "ac4d2fb15c9faaeca852e9f8848af08d"
server_key_iv: "01a9f6168715205339bbb27fe6a47732"
server_key_version: "encryptionkey@0"
I20260926 06:23:51.590408 21720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:23:51.604179 21720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:23:51.605868 21720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:23:51.606026 21720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:23:51.606235 21720 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:23:51.606514 21720 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:23:51.606570 21720 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:23:51.606683 21720 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:23:51.606716 21720 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:23:51.612511 21720 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.131:37353
I20260926 06:23:51.612563 21848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.131:37353 every 8 connection(s)
I20260926 06:23:51.612874 21720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
I20260926 06:23:51.613831 21851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:51.613479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.131:46343)
0926 06:23:51.613630 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:51.613693 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":78,"threads_started":1}
W20260926 06:23:51.614238 21849 heartbeater.cc:644] Failed to heartbeat to 127.4.202.190:41615 (0 consecutive failures): Network error: Failed to ping master at 127.4.202.190:41615: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
I20260926 06:23:51.614467 21851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:51.614397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.131:59349)
0926 06:23:51.614426 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:51.614451 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260926 06:23:51.614799 21851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:51.614710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.131:54411)
0926 06:23:51.614756 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:51.614780 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260926 06:23:51.614925 21849 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 06:23:51.620010 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 21720
I20260926 06:23:52.431656 21853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:52.431433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.129:57527)
0926 06:23:52.431564 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:52.431599 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":58,"threads_started":1}
I20260926 06:23:52.493999 21854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:52.493787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.130:39833)
0926 06:23:52.493922 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:52.493953 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":77,"threads_started":1}
I20260926 06:23:52.615495 21855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:52.615287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.131:39889)
0926 06:23:52.615413 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:52.615448 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":68,"threads_started":1}
I20260926 06:23:52.622834 21856 raft_consensus.cc:482] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:23:52.622978 21856 raft_consensus.cc:504] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515692ec901c470b8b3d3f894cd8bb83" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38333 } } peers { permanent_uuid: "c8900ea32c9e4ca586522252586ec729" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45397 } } peers { permanent_uuid: "a87d2554f9424d85a88fa4556e64ebfe" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37353 } }
I20260926 06:23:52.623310 21856 leader_election.cc:288] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 515692ec901c470b8b3d3f894cd8bb83 (127.4.202.130:38333), a87d2554f9424d85a88fa4556e64ebfe (127.4.202.131:37353)
Sep 26 06:23:52 dist-test-slave-pd0t krb5kdc[20279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403831, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for kudu/127.4.202.130@KRBTEST.COM
Sep 26 06:23:52 dist-test-slave-pd0t krb5kdc[20279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403831, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for kudu/127.4.202.131@KRBTEST.COM
I20260926 06:23:52.635669 21853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:52.623427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:38333 (local address 127.4.202.129:55983)
0926 06:23:52.623504 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:52.623517 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 06:23:52.623577 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:23:52.624573 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:23:52.624576 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:23:52.625086 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:23:52.625347 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:52.625354 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:52.626998 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:52.627000 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:52.628342 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:52.628345 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:52.629210 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:52.629210 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:52.629368 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:52.629989 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:23:52.629999 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:23:52.631860 (+ 1861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:23:52.633985 (+ 2125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:23:52.633991 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:23:52.633993 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:23:52.634276 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:23:52.634519 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:23:52.634520 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:23:52.634520 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:23:52.634677 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:23:52.635128 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:23:52.635138 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:23:52.635276 (+ 138us) client_negotiation.cc:769] Sending connection context
0926 06:23:52.635393 (+ 117us) client_negotiation.cc:240] Negotiation successful
0926 06:23:52.635565 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":94}
I20260926 06:23:52.635669 21858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:52.623507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:55983 (local address 127.4.202.130:38333)
0926 06:23:52.623680 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 06:23:52.623693 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 06:23:52.623729 (+ 36us) server_negotiation.cc:407] Connection header received
0926 06:23:52.624296 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:23:52.624298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:23:52.624358 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:23:52.624519 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:23:52.625481 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:52.626886 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:52.628442 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:52.629076 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:52.629215 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:52.632023 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:23:52.632027 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:23:52.632028 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:23:52.632075 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:23:52.633900 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:23:52.634367 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:23:52.634369 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:23:52.634370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:23:52.634450 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:23:52.634739 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:23:52.634740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:23:52.634740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:23:52.635026 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:23:52.635136 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 06:23:52.635443 (+ 307us) server_negotiation.cc:299] Negotiation successful
0926 06:23:52.635543 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":75,"threads_started":1}
I20260926 06:23:52.635952 21857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:52.623518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:37353 (local address 127.4.202.129:42607)
0926 06:23:52.623660 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:52.623670 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:23:52.623737 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:23:52.624573 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:23:52.624576 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:23:52.625114 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:23:52.625347 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:52.625354 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:52.627008 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:52.627008 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:52.628342 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:23:52.628345 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:23:52.629207 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:23:52.629208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:23:52.629368 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:52.630037 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:23:52.630041 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:23:52.632425 (+ 2384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:23:52.634617 (+ 2192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:23:52.634618 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:23:52.634618 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:23:52.634842 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:23:52.635098 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:23:52.635099 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:23:52.635099 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:23:52.635178 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:23:52.635618 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:23:52.635619 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:23:52.635663 (+ 44us) client_negotiation.cc:769] Sending connection context
0926 06:23:52.635740 (+ 77us) client_negotiation.cc:240] Negotiation successful
0926 06:23:52.635842 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":62,"thread_start_us":58,"threads_started":1}
W20260926 06:23:52.636777 21470 leader_election.cc:341] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 515692ec901c470b8b3d3f894cd8bb83 (127.4.202.130:38333): Not found: Tablet not found: 1ee4a56034c94f5eab503ed58c49df4c
I20260926 06:23:52.636922 21859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:52.623600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:42607 (local address 127.4.202.131:37353)
0926 06:23:52.623728 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0926 06:23:52.623735 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 06:23:52.623762 (+ 27us) server_negotiation.cc:407] Connection header received
0926 06:23:52.624295 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:23:52.624298 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:23:52.624358 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:23:52.624508 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:23:52.625481 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:52.626886 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:52.628442 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:23:52.629090 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:23:52.629196 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:23:52.632537 (+ 3341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:23:52.632541 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:23:52.632541 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:23:52.632552 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:23:52.634496 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:23:52.634921 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:23:52.634923 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:23:52.634924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:23:52.635024 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:23:52.635254 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:23:52.635255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:23:52.635255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:23:52.635539 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:23:52.635616 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 06:23:52.636675 (+ 1059us) server_negotiation.cc:299] Negotiation successful
0926 06:23:52.636811 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":53,"threads_started":1}
W20260926 06:23:52.637553 21469 leader_election.cc:341] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a87d2554f9424d85a88fa4556e64ebfe (127.4.202.131:37353): Not found: Tablet not found: 1ee4a56034c94f5eab503ed58c49df4c
I20260926 06:23:52.637600 21469 leader_election.cc:302] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8900ea32c9e4ca586522252586ec729; no voters: 515692ec901c470b8b3d3f894cd8bb83, a87d2554f9424d85a88fa4556e64ebfe
I20260926 06:23:52.637735 21856 raft_consensus.cc:2726] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 06:23:53.432389 21860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:53.432193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.129:49779)
0926 06:23:53.432311 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:53.432344 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":64,"threads_started":1}
I20260926 06:23:53.494624 21861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:53.494384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.130:35707)
0926 06:23:53.494512 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:53.494559 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
I20260926 06:23:53.616128 21862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:53.615902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.131:38069)
0926 06:23:53.616058 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:53.616092 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":60,"threads_started":1}
I20260926 06:23:54.433025 21863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:54.432815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.129:60579)
0926 06:23:54.432946 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:54.432982 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":58,"threads_started":1}
I20260926 06:23:54.495244 21864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:54.495018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.130:45671)
0926 06:23:54.495173 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:54.495207 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":103,"threads_started":1}
I20260926 06:23:54.616705 21865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:54.616537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.131:49701)
0926 06:23:54.616643 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:54.616671 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":55,"threads_started":1}
I20260926 06:23:54.758387 21866 raft_consensus.cc:482] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:23:54.758476 21866 raft_consensus.cc:504] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515692ec901c470b8b3d3f894cd8bb83" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38333 } } peers { permanent_uuid: "c8900ea32c9e4ca586522252586ec729" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45397 } } peers { permanent_uuid: "a87d2554f9424d85a88fa4556e64ebfe" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37353 } }
I20260926 06:23:54.758708 21866 leader_election.cc:288] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 515692ec901c470b8b3d3f894cd8bb83 (127.4.202.130:38333), a87d2554f9424d85a88fa4556e64ebfe (127.4.202.131:37353)
W20260926 06:23:54.759078 21470 leader_election.cc:341] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 515692ec901c470b8b3d3f894cd8bb83 (127.4.202.130:38333): Not found: Tablet not found: 1ee4a56034c94f5eab503ed58c49df4c
W20260926 06:23:54.759159 21469 leader_election.cc:341] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a87d2554f9424d85a88fa4556e64ebfe (127.4.202.131:37353): Not found: Tablet not found: 1ee4a56034c94f5eab503ed58c49df4c
I20260926 06:23:54.759228 21469 leader_election.cc:302] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8900ea32c9e4ca586522252586ec729; no voters: 515692ec901c470b8b3d3f894cd8bb83, a87d2554f9424d85a88fa4556e64ebfe
I20260926 06:23:54.759301 21866 raft_consensus.cc:2726] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 06:23:55.433712 21867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:55.433477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.129:57051)
0926 06:23:55.433631 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:55.433667 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":62,"threads_started":1}
I20260926 06:23:55.495898 21868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:55.495717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.130:41193)
0926 06:23:55.495831 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:55.495863 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":57,"threads_started":1}
I20260926 06:23:55.617326 21869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:55.617113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.131:42385)
0926 06:23:55.617252 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:55.617288 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":54,"threads_started":1}
I20260926 06:23:56.434491 21870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:56.434268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.129:45949)
0926 06:23:56.434411 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:56.434445 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":76,"threads_started":1}
I20260926 06:23:56.496577 21871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:56.496379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.130:40855)
0926 06:23:56.496507 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:56.496539 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":71,"threads_started":1}
I20260926 06:23:56.617939 21872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:56.617746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.131:52541)
0926 06:23:56.617872 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:56.617905 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":70,"threads_started":1}
I20260926 06:23:57.435263 21873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:57.435066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.129:53013)
0926 06:23:57.435184 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:57.435218 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":64,"threads_started":1}
I20260926 06:23:57.497371 21874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:57.497167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.130:54407)
0926 06:23:57.497292 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:57.497328 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":62,"threads_started":1}
I20260926 06:23:57.618716 21875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:57.618414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.131:35921)
0926 06:23:57.618641 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:57.618680 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":85,"threads_started":1}
I20260926 06:23:58.108332 21876 raft_consensus.cc:482] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:23:58.108426 21876 raft_consensus.cc:504] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515692ec901c470b8b3d3f894cd8bb83" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38333 } } peers { permanent_uuid: "c8900ea32c9e4ca586522252586ec729" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45397 } } peers { permanent_uuid: "a87d2554f9424d85a88fa4556e64ebfe" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37353 } }
I20260926 06:23:58.108654 21876 leader_election.cc:288] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 515692ec901c470b8b3d3f894cd8bb83 (127.4.202.130:38333), a87d2554f9424d85a88fa4556e64ebfe (127.4.202.131:37353)
W20260926 06:23:58.109038 21470 leader_election.cc:341] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 515692ec901c470b8b3d3f894cd8bb83 (127.4.202.130:38333): Not found: Tablet not found: 1ee4a56034c94f5eab503ed58c49df4c
W20260926 06:23:58.109124 21469 leader_election.cc:341] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a87d2554f9424d85a88fa4556e64ebfe (127.4.202.131:37353): Not found: Tablet not found: 1ee4a56034c94f5eab503ed58c49df4c
I20260926 06:23:58.109192 21469 leader_election.cc:302] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8900ea32c9e4ca586522252586ec729; no voters: 515692ec901c470b8b3d3f894cd8bb83, a87d2554f9424d85a88fa4556e64ebfe
I20260926 06:23:58.109290 21876 raft_consensus.cc:2726] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 06:23:58.435956 21877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:58.435726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.129:36259)
0926 06:23:58.435873 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:58.435910 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":75,"threads_started":1}
I20260926 06:23:58.498175 21878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:58.497934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.130:49583)
0926 06:23:58.498100 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:58.498136 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":99,"threads_started":1}
I20260926 06:23:58.619354 21879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:58.619132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.131:54335)
0926 06:23:58.619280 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:58.619316 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":54,"threads_started":1}
I20260926 06:23:59.436647 21880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:59.436443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.129:44107)
0926 06:23:59.436565 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:59.436602 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":63,"threads_started":1}
I20260926 06:23:59.498891 21881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:59.498707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.130:56979)
0926 06:23:59.498817 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:59.498851 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":52,"threads_started":1}
I20260926 06:23:59.619998 21882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:23:59.619819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.131:47283)
0926 06:23:59.619931 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0926 06:23:59.619963 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":57,"threads_started":1}
I20260926 06:24:00.437294 21883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:00.437097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.129:60331)
0926 06:24:00.437215 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:00.437248 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
I20260926 06:24:00.499454 21884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:00.499261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.130:35979)
0926 06:24:00.499384 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:00.499415 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":70,"threads_started":1}
I20260926 06:24:00.620684 21885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:00.620504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.131:43923)
0926 06:24:00.620617 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:00.620649 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":71,"threads_started":1}
I20260926 06:24:01.158429 21886 raft_consensus.cc:482] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:24:01.158510 21886 raft_consensus.cc:504] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515692ec901c470b8b3d3f894cd8bb83" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38333 } } peers { permanent_uuid: "c8900ea32c9e4ca586522252586ec729" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 45397 } } peers { permanent_uuid: "a87d2554f9424d85a88fa4556e64ebfe" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37353 } }
I20260926 06:24:01.158686 21886 leader_election.cc:288] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 515692ec901c470b8b3d3f894cd8bb83 (127.4.202.130:38333), a87d2554f9424d85a88fa4556e64ebfe (127.4.202.131:37353)
W20260926 06:24:01.159019 21469 leader_election.cc:341] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a87d2554f9424d85a88fa4556e64ebfe (127.4.202.131:37353): Not found: Tablet not found: 1ee4a56034c94f5eab503ed58c49df4c
W20260926 06:24:01.159096 21470 leader_election.cc:341] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 515692ec901c470b8b3d3f894cd8bb83 (127.4.202.130:38333): Not found: Tablet not found: 1ee4a56034c94f5eab503ed58c49df4c
I20260926 06:24:01.159166 21470 leader_election.cc:302] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8900ea32c9e4ca586522252586ec729; no voters: 515692ec901c470b8b3d3f894cd8bb83, a87d2554f9424d85a88fa4556e64ebfe
I20260926 06:24:01.159247 21886 raft_consensus.cc:2726] T 1ee4a56034c94f5eab503ed58c49df4c P c8900ea32c9e4ca586522252586ec729 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 06:24:01.438108 21887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:01.437879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.129:33751)
0926 06:24:01.438011 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:01.438062 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":61,"threads_started":1}
I20260926 06:24:01.500168 21888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:01.499911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.130:50773)
0926 06:24:01.500056 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:01.500105 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":57,"threads_started":1}
I20260926 06:24:01.621541 21889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:01.621291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.131:41867)
0926 06:24:01.621442 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:01.621505 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":97,"threads_started":1}
I20260926 06:24:02.438870 21890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:02.438652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.129:40383)
0926 06:24:02.438788 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:02.438824 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":58,"threads_started":1}
I20260926 06:24:02.500929 21891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:02.500710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.130:53149)
0926 06:24:02.500855 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:02.500887 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":76,"threads_started":1}
I20260926 06:24:02.622073 21892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:02.621905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.131:36157)
0926 06:24:02.622009 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:02.622037 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":56,"threads_started":1}
I20260926 06:24:03.439567 21893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:03.439361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.129:50299)
0926 06:24:03.439480 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:03.439521 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":56,"threads_started":1}
I20260926 06:24:03.501555 21894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:03.501355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.130:59031)
0926 06:24:03.501478 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:03.501514 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":52,"threads_started":1}
I20260926 06:24:03.622673 21895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:03.622412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.131:37521)
0926 06:24:03.622541 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:03.622622 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":55,"threads_started":1}
I20260926 06:24:04.440222 21896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:04.440024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.129:41321)
0926 06:24:04.440142 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:04.440175 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":63,"threads_started":1}
I20260926 06:24:04.502216 21897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:04.502000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.130:39975)
0926 06:24:04.502135 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:04.502170 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":75,"threads_started":1}
I20260926 06:24:04.623240 21898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:04.623060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.131:43573)
0926 06:24:04.623173 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:04.623203 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":57,"threads_started":1}
I20260926 06:24:05.440860 21899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:05.440669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.129:42347)
0926 06:24:05.440783 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:05.440815 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":57,"threads_started":1}
I20260926 06:24:05.502990 21900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:05.502807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.130:38273)
0926 06:24:05.502913 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:05.502946 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":53,"threads_started":1}
I20260926 06:24:05.623783 21901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:05.623597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.131:40685)
0926 06:24:05.623717 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:05.623746 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":70,"threads_started":1}
I20260926 06:24:06.441581 21902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:06.441387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.129:36153)
0926 06:24:06.441506 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:06.441538 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":60,"threads_started":1}
I20260926 06:24:06.503728 21903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:06.503538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.130:40847)
0926 06:24:06.503650 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:06.503684 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":58,"threads_started":1}
I20260926 06:24:06.624507 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:06.624294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:41615 (local address 127.4.202.131:46769)
0926 06:24:06.624429 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:06.624466 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.190:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":62,"threads_started":1}
I20260926 06:24:06.629664 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 21420
I20260926 06:24:06.634387 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 21587
I20260926 06:24:06.638484 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 21720
2026-09-26T06:24:06Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58686 ms)
[----------] 30 tests from SecurityITest (255007 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:24:06 dist-test-slave-pd0t krb5kdc[21907](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:24:06 dist-test-slave-pd0t krb5kdc[21907](info): set up 2 sockets
Sep 26 06:24:06 dist-test-slave-pd0t krb5kdc[21907](info): commencing operation
krb5kdc: starting...
W20260926 06:24:07.678272 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:24:07 dist-test-slave-pd0t krb5kdc[21907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:24:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:24:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.202.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
I20260926 06:24:07.793200 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:42137
--webserver_interface=127.4.202.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:44453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:42137
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:24:07.871452 21920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:07.871662 21920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:07.871697 21920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:07.872838 21920 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:24:07.872895 21920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:07.872920 21920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:07.872938 21920 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:24:07.872972 21920 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:24:07.874387 21920 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:44453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:42137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:42137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:07.874675 21920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:07.874888 21920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:07.879768 21926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:07.879761 21928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:07.879760 21925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:07.879936 21920 server_base.cc:1034] running on GCE node
I20260926 06:24:07.880174 21920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:07.880482 21920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:07.881636 21920 hybrid_clock.cc:648] HybridClock initialized: now 1790403847881617 us; error 35 us; skew 500 ppm
Sep 26 06:24:07 dist-test-slave-pd0t krb5kdc[21907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403847, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:07.883999 21920 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:24:07.884413 21920 webserver.cc:460] Webserver started at http://127.4.202.190:41503/ using document root <none> and password file <none>
I20260926 06:24:07.884620 21920 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:07.884682 21920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:07.884819 21920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:07.885776 21920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "34c5dde05928457cb0ca7a3959b9923c"
format_stamp: "Formatted at 2026-09-26 06:24:07 on dist-test-slave-pd0t"
server_key: "b83b9bd925023642d950a2b211943b74"
server_key_iv: "e4518846834fcccea8377641a91572f3"
server_key_version: "encryptionkey@0"
I20260926 06:24:07.886124 21920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "34c5dde05928457cb0ca7a3959b9923c"
format_stamp: "Formatted at 2026-09-26 06:24:07 on dist-test-slave-pd0t"
server_key: "b83b9bd925023642d950a2b211943b74"
server_key_iv: "e4518846834fcccea8377641a91572f3"
server_key_version: "encryptionkey@0"
I20260926 06:24:07.887495 21920 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:24:07.888160 21935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:07.888317 21920 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:24:07.888414 21920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "34c5dde05928457cb0ca7a3959b9923c"
format_stamp: "Formatted at 2026-09-26 06:24:07 on dist-test-slave-pd0t"
server_key: "b83b9bd925023642d950a2b211943b74"
server_key_iv: "e4518846834fcccea8377641a91572f3"
server_key_version: "encryptionkey@0"
I20260926 06:24:07.888486 21920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:07.915418 21920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:07.917340 21920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:07.917501 21920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:07.921427 21920 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:42137
I20260926 06:24:07.921461 21987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:42137 every 8 connection(s)
I20260926 06:24:07.921792 21920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:24:07.922698 21988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:07.924907 21988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34c5dde05928457cb0ca7a3959b9923c: Bootstrap starting.
I20260926 06:24:07.925496 21988 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34c5dde05928457cb0ca7a3959b9923c: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:07.925772 21988 log.cc:824] T 00000000000000000000000000000000 P 34c5dde05928457cb0ca7a3959b9923c: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:07.926497 21988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34c5dde05928457cb0ca7a3959b9923c: No bootstrap required, opened a new log
I20260926 06:24:07.927523 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 21920
I20260926 06:24:07.927657 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:24:07.927847 4906 external_mini_cluster.cc:1428] Setting key 9211b1f30f281c68f37a88983bbe115e
I20260926 06:24:07.928663 21988 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34c5dde05928457cb0ca7a3959b9923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c5dde05928457cb0ca7a3959b9923c" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42137 } }
I20260926 06:24:07.928824 21988 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34c5dde05928457cb0ca7a3959b9923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:07.928923 21988 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34c5dde05928457cb0ca7a3959b9923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34c5dde05928457cb0ca7a3959b9923c, State: Initialized, Role: FOLLOWER
I20260926 06:24:07.929085 21988 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34c5dde05928457cb0ca7a3959b9923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c5dde05928457cb0ca7a3959b9923c" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42137 } }
I20260926 06:24:07.929172 21988 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34c5dde05928457cb0ca7a3959b9923c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:24:07.929204 21988 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34c5dde05928457cb0ca7a3959b9923c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:24:07.929258 21988 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34c5dde05928457cb0ca7a3959b9923c [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:07.930157 21988 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34c5dde05928457cb0ca7a3959b9923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c5dde05928457cb0ca7a3959b9923c" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42137 } }
I20260926 06:24:07.930249 21988 leader_election.cc:302] T 00000000000000000000000000000000 P 34c5dde05928457cb0ca7a3959b9923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34c5dde05928457cb0ca7a3959b9923c; no voters:
I20260926 06:24:07.930459 21988 leader_election.cc:288] T 00000000000000000000000000000000 P 34c5dde05928457cb0ca7a3959b9923c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:24:07.930490 21993 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34c5dde05928457cb0ca7a3959b9923c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:24:07.930660 21993 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34c5dde05928457cb0ca7a3959b9923c [term 1 LEADER]: Becoming Leader. State: Replica: 34c5dde05928457cb0ca7a3959b9923c, State: Running, Role: LEADER
I20260926 06:24:07.930721 21988 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34c5dde05928457cb0ca7a3959b9923c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:24:07.930819 21993 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34c5dde05928457cb0ca7a3959b9923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c5dde05928457cb0ca7a3959b9923c" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42137 } }
I20260926 06:24:07.931198 21993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34c5dde05928457cb0ca7a3959b9923c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34c5dde05928457cb0ca7a3959b9923c. Latest consensus state: current_term: 1 leader_uuid: "34c5dde05928457cb0ca7a3959b9923c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c5dde05928457cb0ca7a3959b9923c" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42137 } } }
I20260926 06:24:07.931476 21993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34c5dde05928457cb0ca7a3959b9923c [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:07.931383 21994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34c5dde05928457cb0ca7a3959b9923c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34c5dde05928457cb0ca7a3959b9923c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c5dde05928457cb0ca7a3959b9923c" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 42137 } } }
I20260926 06:24:07.931890 21994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34c5dde05928457cb0ca7a3959b9923c [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:07.931874 22002 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:24:07.932679 22002 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:24:07.934365 22002 catalog_manager.cc:1347] Generated new cluster ID: 0875fe0859e64d9aa94cc5759789ad3e
I20260926 06:24:07.934414 22002 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:24:07 dist-test-slave-pd0t krb5kdc[21907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:24:07.938848 21992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:07.928346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36758 (local address 127.4.202.190:42137)
0926 06:24:07.928520 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:07.928528 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:07.928559 (+ 31us) server_negotiation.cc:407] Connection header received
0926 06:24:07.928649 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:07.928654 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:07.928693 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:07.928792 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:07.929604 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:07.930615 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:07.931697 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:07.932356 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:07.932458 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:07.935326 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:07.935330 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:07.935330 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:07.935341 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:07.937247 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:07.937610 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:07.937613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:07.937613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:07.937743 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:07.937934 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:07.937935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:07.937935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:07.938232 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:07.938385 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:07.938475 (+ 90us) server_negotiation.cc:299] Negotiation successful
0926 06:24:07.938634 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":96,"threads_started":1}
I20260926 06:24:07.940013 22002 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:24:07.940428 22002 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:24:07.943053 22002 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34c5dde05928457cb0ca7a3959b9923c: Generated new TSK 0
I20260926 06:24:07.943187 22002 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.202.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
I20260926 06:24:07.968954 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.129:0
--local_ip_for_outbound_sockets=127.4.202.129
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:42137
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:44453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:24:08.048138 22014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:08.048357 22014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:08.048393 22014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:08.049553 22014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:08.049616 22014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:08.049659 22014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.129
I20260926 06:24:08.051137 22014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:44453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:42137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22014
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.129
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:08.051376 22014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:08.051591 22014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:08.057391 22019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:08.057372 22022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:08.057386 22020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:08.057830 22014 server_base.cc:1034] running on GCE node
I20260926 06:24:08.058053 22014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:08.058296 22014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:08.059456 22014 hybrid_clock.cc:648] HybridClock initialized: now 1790403848059435 us; error 37 us; skew 500 ppm
Sep 26 06:24:08 dist-test-slave-pd0t krb5kdc[21907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403848, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:08.061856 22014 init.cc:377] Logged in from keytab as kudu/127.4.202.129@KRBTEST.COM (short username kudu)
I20260926 06:24:08.062280 22014 webserver.cc:460] Webserver started at http://127.4.202.129:33979/ using document root <none> and password file <none>
I20260926 06:24:08.062537 22014 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:08.062633 22014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:08.062767 22014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:08.063817 22014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-0/data/instance:
uuid: "a4bec2b8a6754c4eba7a45b19691e082"
format_stamp: "Formatted at 2026-09-26 06:24:08 on dist-test-slave-pd0t"
server_key: "d75b7c9925de2e7caecc8d7f0cce3e1f"
server_key_iv: "15a4cf6a7f003b95ddb28736b6f7d3a2"
server_key_version: "encryptionkey@0"
I20260926 06:24:08.064173 22014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance:
uuid: "a4bec2b8a6754c4eba7a45b19691e082"
format_stamp: "Formatted at 2026-09-26 06:24:08 on dist-test-slave-pd0t"
server_key: "d75b7c9925de2e7caecc8d7f0cce3e1f"
server_key_iv: "15a4cf6a7f003b95ddb28736b6f7d3a2"
server_key_version: "encryptionkey@0"
I20260926 06:24:08.065456 22014 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:24:08.066135 22029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:08.066314 22014 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:24:08.066409 22014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-0/wal
uuid: "a4bec2b8a6754c4eba7a45b19691e082"
format_stamp: "Formatted at 2026-09-26 06:24:08 on dist-test-slave-pd0t"
server_key: "d75b7c9925de2e7caecc8d7f0cce3e1f"
server_key_iv: "15a4cf6a7f003b95ddb28736b6f7d3a2"
server_key_version: "encryptionkey@0"
I20260926 06:24:08.066474 22014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:08.077912 22014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:08.079780 22014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:08.079928 22014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:08.080144 22014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:24:08.080436 22014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:24:08.080487 22014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:08.080520 22014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:24:08.080545 22014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:08.086172 22014 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.129:39543
I20260926 06:24:08.086244 22142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.129:39543 every 8 connection(s)
I20260926 06:24:08.086539 22014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
Sep 26 06:24:08 dist-test-slave-pd0t krb5kdc[21907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403848, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:24:08.093676 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 22014
I20260926 06:24:08.093796 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance
I20260926 06:24:08.094007 4906 external_mini_cluster.cc:1428] Setting key fd7156b30ff4045684e6a75526e41435
I20260926 06:24:08.095481 22145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:08.087152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42137 (local address 127.4.202.129:58429)
0926 06:24:08.087313 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:08.087352 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:08.087819 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:08.088048 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:08.088052 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:08.088602 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:08.088787 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:08.088791 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.089422 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.089423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:08.090230 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:08.090232 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.090675 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.090676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:08.090805 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:08.091257 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:24:08.091265 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:08.092572 (+ 1307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:08.094038 (+ 1466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:08.094046 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:08.094048 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:08.094357 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:08.094543 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:08.094544 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:08.094544 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:08.094705 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:08.094997 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:08.095001 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:08.095128 (+ 127us) client_negotiation.cc:769] Sending connection context
0926 06:24:08.095211 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 06:24:08.095326 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":88,"threads_started":1}
I20260926 06:24:08.095517 21992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:08.087197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:58429 (local address 127.4.202.190:42137)
0926 06:24:08.087242 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:08.087245 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:08.087434 (+ 189us) server_negotiation.cc:407] Connection header received
0926 06:24:08.087899 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:08.087900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:08.087913 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:08.087970 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:08.088863 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.089345 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.090329 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.090560 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.090667 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:08.092664 (+ 1997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:08.092666 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:08.092666 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:08.092676 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:08.093959 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:08.094423 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:08.094424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:08.094425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:08.094478 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:08.094780 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:08.094781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:08.094781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:08.094926 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:08.095003 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:08.095256 (+ 253us) server_negotiation.cc:299] Negotiation successful
0926 06:24:08.095365 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 06:24:08.095997 22143 heartbeater.cc:347] Connected to a master server at 127.4.202.190:42137
I20260926 06:24:08.096114 22143 heartbeater.cc:464] Registering TS with master...
I20260926 06:24:08.096288 22143 heartbeater.cc:511] Master 127.4.202.190:42137 requested a full tablet report, sending...
I20260926 06:24:08.096807 21952 ts_manager.cc:194] Registered new tserver with Master: a4bec2b8a6754c4eba7a45b19691e082 (127.4.202.129:39543)
I20260926 06:24:08.097389 21952 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.129@KRBTEST.COM'} at 127.4.202.129:58429
WARNING: no policy specified for kudu/127.4.202.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
I20260926 06:24:08.117512 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.130:0
--local_ip_for_outbound_sockets=127.4.202.130
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:42137
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:44453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:24:08.196790 22148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:08.197016 22148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:08.197041 22148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:08.198210 22148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:08.198251 22148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:08.198350 22148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.130
I20260926 06:24:08.200026 22148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:44453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:42137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.130
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:08.200260 22148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:08.200495 22148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:08.205240 22156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:08.205366 22154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:08.205358 22148 server_base.cc:1034] running on GCE node
W20260926 06:24:08.205446 22153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:08.205677 22148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:08.205909 22148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:08.207050 22148 hybrid_clock.cc:648] HybridClock initialized: now 1790403848207040 us; error 29 us; skew 500 ppm
Sep 26 06:24:08 dist-test-slave-pd0t krb5kdc[21907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403848, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:08.209342 22148 init.cc:377] Logged in from keytab as kudu/127.4.202.130@KRBTEST.COM (short username kudu)
I20260926 06:24:08.209730 22148 webserver.cc:460] Webserver started at http://127.4.202.130:38207/ using document root <none> and password file <none>
I20260926 06:24:08.209955 22148 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:08.210024 22148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:08.210155 22148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:08.211169 22148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-1/data/instance:
uuid: "b8602f6ef0814ad28fb9f1f2082c7fbd"
format_stamp: "Formatted at 2026-09-26 06:24:08 on dist-test-slave-pd0t"
server_key: "c9f91aaaf1a14a7f30c1337b22765196"
server_key_iv: "912a11725e26cbea3f2074a6be6e0565"
server_key_version: "encryptionkey@0"
I20260926 06:24:08.211501 22148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance:
uuid: "b8602f6ef0814ad28fb9f1f2082c7fbd"
format_stamp: "Formatted at 2026-09-26 06:24:08 on dist-test-slave-pd0t"
server_key: "c9f91aaaf1a14a7f30c1337b22765196"
server_key_iv: "912a11725e26cbea3f2074a6be6e0565"
server_key_version: "encryptionkey@0"
I20260926 06:24:08.212867 22148 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:24:08.213577 22163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:08.213742 22148 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:24:08.213827 22148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-1/wal
uuid: "b8602f6ef0814ad28fb9f1f2082c7fbd"
format_stamp: "Formatted at 2026-09-26 06:24:08 on dist-test-slave-pd0t"
server_key: "c9f91aaaf1a14a7f30c1337b22765196"
server_key_iv: "912a11725e26cbea3f2074a6be6e0565"
server_key_version: "encryptionkey@0"
I20260926 06:24:08.213896 22148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:08.230384 22148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:08.232784 22148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:08.232933 22148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:08.233140 22148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:24:08.233428 22148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:24:08.233460 22148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:08.233482 22148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:24:08.233494 22148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:08.239189 22148 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.130:41957
I20260926 06:24:08.239257 22276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.130:41957 every 8 connection(s)
I20260926 06:24:08.239566 22148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
I20260926 06:24:08.241621 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 22148
I20260926 06:24:08.241780 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance
I20260926 06:24:08.242076 4906 external_mini_cluster.cc:1428] Setting key e3d33080db8b60551aeb1951085c7bbc
Sep 26 06:24:08 dist-test-slave-pd0t krb5kdc[21907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403848, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:24:08.249492 22279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:08.240289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42137 (local address 127.4.202.130:47451)
0926 06:24:08.240440 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:08.240480 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:08.240931 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:08.241131 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:08.241134 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:08.241728 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:08.241966 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:08.241970 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.242916 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.242917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:08.243696 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:08.243698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.244201 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.244202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:08.244331 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:08.244831 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:24:08.244839 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:08.246380 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:08.248227 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:08.248233 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:08.248235 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:08.248445 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:08.248654 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:08.248655 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:08.248655 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:08.248752 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:08.249045 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:08.249049 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:08.249154 (+ 105us) client_negotiation.cc:769] Sending connection context
0926 06:24:08.249234 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 06:24:08.249345 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
I20260926 06:24:08.249483 21992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:08.240307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:47451 (local address 127.4.202.190:42137)
0926 06:24:08.240366 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:08.240368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:08.240579 (+ 211us) server_negotiation.cc:407] Connection header received
0926 06:24:08.241006 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:08.241007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:08.241018 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:08.241083 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:08.242061 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.242843 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.243786 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.244108 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.244181 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:08.246469 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:08.246471 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:08.246472 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:08.246479 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:08.248146 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:08.248519 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:08.248520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:08.248521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:08.248583 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:08.248816 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:08.248817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:08.248817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:08.248968 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:08.249040 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:08.249280 (+ 240us) server_negotiation.cc:299] Negotiation successful
0926 06:24:08.249371 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 06:24:08.249971 22277 heartbeater.cc:347] Connected to a master server at 127.4.202.190:42137
I20260926 06:24:08.250067 22277 heartbeater.cc:464] Registering TS with master...
I20260926 06:24:08.250263 22277 heartbeater.cc:511] Master 127.4.202.190:42137 requested a full tablet report, sending...
I20260926 06:24:08.250687 21952 ts_manager.cc:194] Registered new tserver with Master: b8602f6ef0814ad28fb9f1f2082c7fbd (127.4.202.130:41957)
WARNING: no policy specified for kudu/127.4.202.131@KRBTEST.COM; defaulting to no policy
I20260926 06:24:08.251086 21952 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.130@KRBTEST.COM'} at 127.4.202.130:47451
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
I20260926 06:24:08.268016 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.131:0
--local_ip_for_outbound_sockets=127.4.202.131
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:42137
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:44453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:24:08.347851 22282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:08.348035 22282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:08.348058 22282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:08.349104 22282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:08.349140 22282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:08.349166 22282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.131
I20260926 06:24:08.350512 22282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:44453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:42137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.131
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:08.350760 22282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:08.350934 22282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:08.355674 22287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:08.355715 22290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:08.355842 22282 server_base.cc:1034] running on GCE node
W20260926 06:24:08.355947 22288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:08.356134 22282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:08.356316 22282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:08.357468 22282 hybrid_clock.cc:648] HybridClock initialized: now 1790403848357459 us; error 34 us; skew 500 ppm
Sep 26 06:24:08 dist-test-slave-pd0t krb5kdc[21907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403848, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:08.359728 22282 init.cc:377] Logged in from keytab as kudu/127.4.202.131@KRBTEST.COM (short username kudu)
I20260926 06:24:08.360074 22282 webserver.cc:460] Webserver started at http://127.4.202.131:32865/ using document root <none> and password file <none>
I20260926 06:24:08.360247 22282 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:08.360282 22282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:08.360368 22282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:08.361236 22282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-2/data/instance:
uuid: "82cec124bea34379b3741f49b809316a"
format_stamp: "Formatted at 2026-09-26 06:24:08 on dist-test-slave-pd0t"
server_key: "1819fe586568b79c90652c2445e95ff4"
server_key_iv: "32476778bfb08e183e81a04aa323b1c3"
server_key_version: "encryptionkey@0"
I20260926 06:24:08.361503 22282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance:
uuid: "82cec124bea34379b3741f49b809316a"
format_stamp: "Formatted at 2026-09-26 06:24:08 on dist-test-slave-pd0t"
server_key: "1819fe586568b79c90652c2445e95ff4"
server_key_iv: "32476778bfb08e183e81a04aa323b1c3"
server_key_version: "encryptionkey@0"
I20260926 06:24:08.362764 22282 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:24:08.363430 22297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:08.363613 22282 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:24:08.363706 22282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-2/wal
uuid: "82cec124bea34379b3741f49b809316a"
format_stamp: "Formatted at 2026-09-26 06:24:08 on dist-test-slave-pd0t"
server_key: "1819fe586568b79c90652c2445e95ff4"
server_key_iv: "32476778bfb08e183e81a04aa323b1c3"
server_key_version: "encryptionkey@0"
I20260926 06:24:08.363773 22282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:08.372025 22282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:08.373756 22282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:08.373901 22282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:08.374117 22282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:24:08.374379 22282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:24:08.374411 22282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:08.374431 22282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:24:08.374475 22282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:08.380044 22282 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.131:43723
I20260926 06:24:08.380128 22410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.131:43723 every 8 connection(s)
I20260926 06:24:08.380383 22282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
I20260926 06:24:08.382261 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 22282
I20260926 06:24:08.382367 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance
I20260926 06:24:08.382560 4906 external_mini_cluster.cc:1428] Setting key 3233d4724f429db6ba4f060e6fc375de
Sep 26 06:24:08 dist-test-slave-pd0t krb5kdc[21907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403848, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:24:08.388856 21992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:08.381020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:34907 (local address 127.4.202.190:42137)
0926 06:24:08.381064 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:08.381066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:08.381227 (+ 161us) server_negotiation.cc:407] Connection header received
0926 06:24:08.381698 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:08.381699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:08.381711 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:08.381773 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:08.382951 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.383634 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.384542 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.384720 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.384768 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:08.386646 (+ 1878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:08.386648 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:08.386648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:08.386655 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:08.387737 (+ 1082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:08.388123 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:08.388124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:08.388125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:08.388186 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:08.388395 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:08.388396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:08.388396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:08.388491 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:08.388541 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:08.388741 (+ 200us) server_negotiation.cc:299] Negotiation successful
0926 06:24:08.388796 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 06:24:08.389037 22413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:08.380976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:42137 (local address 127.4.202.131:34907)
0926 06:24:08.381107 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:08.381151 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:08.381601 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:08.381837 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:08.381841 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:08.382571 (+ 730us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:08.382853 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:08.382858 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.383703 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.383704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:08.384472 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:08.384474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.384783 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.384784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:08.384877 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:08.385285 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:24:08.385293 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:08.386520 (+ 1227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:08.387820 (+ 1300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:08.387825 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:08.387827 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:08.388050 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:08.388254 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:08.388254 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:08.388255 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:08.388344 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:08.388541 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:08.388544 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:08.388642 (+ 98us) client_negotiation.cc:769] Sending connection context
0926 06:24:08.388717 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 06:24:08.388821 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":61,"threads_started":1}
I20260926 06:24:08.389421 22411 heartbeater.cc:347] Connected to a master server at 127.4.202.190:42137
I20260926 06:24:08.389504 22411 heartbeater.cc:464] Registering TS with master...
I20260926 06:24:08.389632 22411 heartbeater.cc:511] Master 127.4.202.190:42137 requested a full tablet report, sending...
I20260926 06:24:08.389967 21952 ts_manager.cc:194] Registered new tserver with Master: 82cec124bea34379b3741f49b809316a (127.4.202.131:43723)
I20260926 06:24:08.390288 21952 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.131@KRBTEST.COM'} at 127.4.202.131:34907
I20260926 06:24:08.394265 4906 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:24:08.401078 21992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:08.395647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36774 (local address 127.4.202.190:42137)
0926 06:24:08.395720 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:08.395722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:08.395800 (+ 78us) server_negotiation.cc:407] Connection header received
0926 06:24:08.395849 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:08.395849 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:08.395857 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:08.395912 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:08.396435 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.396892 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.397604 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.397772 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.397827 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:08.398959 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:08.398960 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:08.398961 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:08.398966 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:08.400165 (+ 1199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:08.400472 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:08.400473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:08.400473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:08.400526 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:08.400686 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:08.400686 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:08.400687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:08.400809 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:08.400889 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:08.400965 (+ 76us) server_negotiation.cc:299] Negotiation successful
0926 06:24:08.401022 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 06:24:08.402706 21952 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36774:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:24:08.404189 21952 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:24:08.410089 22420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:08.406132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:41957 (local address 127.0.0.1:46100)
0926 06:24:08.406379 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:08.406417 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:08.406507 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:08.406754 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:08.406756 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:08.406758 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:24:08.406814 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:08.406815 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.407868 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.407869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:08.409007 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:08.409010 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.409801 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.409802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:08.409878 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:08.409891 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 06:24:08.409955 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 06:24:08.410008 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":81,"threads_started":1}
I20260926 06:24:08.410140 22423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:08.406362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46100 (local address 127.4.202.130:41957)
0926 06:24:08.406523 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:08.406528 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:08.406542 (+ 14us) server_negotiation.cc:407] Connection header received
0926 06:24:08.406616 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:08.406618 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:08.406634 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:08.406711 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:24:08.407019 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.407771 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.409105 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.409719 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.409800 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:08.409856 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:08.410014 (+ 158us) server_negotiation.cc:299] Negotiation successful
0926 06:24:08.410057 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":83,"threads_started":1}
I20260926 06:24:08.410218 22422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:08.406133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:39543 (local address 127.0.0.1:35308)
0926 06:24:08.406542 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:08.406553 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:08.406626 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:08.406961 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:08.406962 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:08.406965 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:24:08.407012 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:08.407013 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.408302 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.408303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:08.409175 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:08.409177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.410012 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.410014 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:08.410076 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:08.410079 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:24:08.410109 (+ 30us) client_negotiation.cc:240] Negotiation successful
0926 06:24:08.410145 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"mutex_wait_us":4,"spinlock_wait_cycles":1408,"thread_start_us":81,"threads_started":1}
I20260926 06:24:08.410514 22421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:08.406275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35308 (local address 127.4.202.129:39543)
0926 06:24:08.406749 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:08.406753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:08.406761 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:24:08.406798 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:08.406802 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:08.406817 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:08.406883 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:24:08.407095 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.408186 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.409252 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.409913 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.410286 (+ 373us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:08.410354 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:08.410380 (+ 26us) server_negotiation.cc:299] Negotiation successful
0926 06:24:08.410424 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":89,"threads_started":1}
I20260926 06:24:08.410887 22419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:08.406348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:43723 (local address 127.0.0.1:46288)
0926 06:24:08.406629 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:08.406638 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:08.406679 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:08.406865 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:08.406865 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:08.406867 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:24:08.406907 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:08.406908 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.408281 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.408281 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:08.409402 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:08.409405 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.410676 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.410677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:08.410739 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:08.410743 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:24:08.410791 (+ 48us) client_negotiation.cc:240] Negotiation successful
0926 06:24:08.410825 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":29,"thread_start_us":66,"threads_started":1}
I20260926 06:24:08.411020 22211 tablet_service.cc:1467] Processing CreateTablet for tablet 65d70af6a1934a0486c0a45ab062e808 (DEFAULT_TABLE table=test-table [id=fa906790a297428c98ea2585f0d24d1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:24:08.411170 22077 tablet_service.cc:1467] Processing CreateTablet for tablet 65d70af6a1934a0486c0a45ab062e808 (DEFAULT_TABLE table=test-table [id=fa906790a297428c98ea2585f0d24d1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:24:08.411372 22211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65d70af6a1934a0486c0a45ab062e808. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:08.411417 22424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:08.406520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46288 (local address 127.4.202.131:43723)
0926 06:24:08.406706 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:08.406710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:08.406720 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:24:08.406761 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:08.406763 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:08.406780 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:08.406855 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:24:08.407081 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.408159 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.409489 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.410611 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.410675 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:08.410722 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:08.410984 (+ 262us) server_negotiation.cc:299] Negotiation successful
0926 06:24:08.411021 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":97,"threads_started":1}
I20260926 06:24:08.411500 22077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65d70af6a1934a0486c0a45ab062e808. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:08.413933 22425 tablet_bootstrap.cc:492] T 65d70af6a1934a0486c0a45ab062e808 P b8602f6ef0814ad28fb9f1f2082c7fbd: Bootstrap starting.
I20260926 06:24:08.414544 22426 tablet_bootstrap.cc:492] T 65d70af6a1934a0486c0a45ab062e808 P a4bec2b8a6754c4eba7a45b19691e082: Bootstrap starting.
I20260926 06:24:08.414685 22425 tablet_bootstrap.cc:654] T 65d70af6a1934a0486c0a45ab062e808 P b8602f6ef0814ad28fb9f1f2082c7fbd: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:08.414657 22345 tablet_service.cc:1467] Processing CreateTablet for tablet 65d70af6a1934a0486c0a45ab062e808 (DEFAULT_TABLE table=test-table [id=fa906790a297428c98ea2585f0d24d1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:24:08.414929 22345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65d70af6a1934a0486c0a45ab062e808. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:08.415252 22425 log.cc:824] T 65d70af6a1934a0486c0a45ab062e808 P b8602f6ef0814ad28fb9f1f2082c7fbd: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:08.415294 22426 tablet_bootstrap.cc:654] T 65d70af6a1934a0486c0a45ab062e808 P a4bec2b8a6754c4eba7a45b19691e082: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:08.415798 22426 log.cc:824] T 65d70af6a1934a0486c0a45ab062e808 P a4bec2b8a6754c4eba7a45b19691e082: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:08.416029 22425 tablet_bootstrap.cc:492] T 65d70af6a1934a0486c0a45ab062e808 P b8602f6ef0814ad28fb9f1f2082c7fbd: No bootstrap required, opened a new log
I20260926 06:24:08.416080 22425 ts_tablet_manager.cc:1397] T 65d70af6a1934a0486c0a45ab062e808 P b8602f6ef0814ad28fb9f1f2082c7fbd: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 06:24:08.416522 22426 tablet_bootstrap.cc:492] T 65d70af6a1934a0486c0a45ab062e808 P a4bec2b8a6754c4eba7a45b19691e082: No bootstrap required, opened a new log
I20260926 06:24:08.416605 22426 ts_tablet_manager.cc:1397] T 65d70af6a1934a0486c0a45ab062e808 P a4bec2b8a6754c4eba7a45b19691e082: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:24:08.417657 22429 tablet_bootstrap.cc:492] T 65d70af6a1934a0486c0a45ab062e808 P 82cec124bea34379b3741f49b809316a: Bootstrap starting.
I20260926 06:24:08.418059 22425 raft_consensus.cc:348] T 65d70af6a1934a0486c0a45ab062e808 P b8602f6ef0814ad28fb9f1f2082c7fbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8602f6ef0814ad28fb9f1f2082c7fbd" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 41957 } } peers { permanent_uuid: "82cec124bea34379b3741f49b809316a" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 43723 } } peers { permanent_uuid: "a4bec2b8a6754c4eba7a45b19691e082" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 39543 } }
I20260926 06:24:08.418241 22425 raft_consensus.cc:374] T 65d70af6a1934a0486c0a45ab062e808 P b8602f6ef0814ad28fb9f1f2082c7fbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:08.418231 22429 tablet_bootstrap.cc:654] T 65d70af6a1934a0486c0a45ab062e808 P 82cec124bea34379b3741f49b809316a: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:08.418274 22425 raft_consensus.cc:729] T 65d70af6a1934a0486c0a45ab062e808 P b8602f6ef0814ad28fb9f1f2082c7fbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8602f6ef0814ad28fb9f1f2082c7fbd, State: Initialized, Role: FOLLOWER
I20260926 06:24:08.418418 22425 consensus_queue.cc:260] T 65d70af6a1934a0486c0a45ab062e808 P b8602f6ef0814ad28fb9f1f2082c7fbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8602f6ef0814ad28fb9f1f2082c7fbd" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 41957 } } peers { permanent_uuid: "82cec124bea34379b3741f49b809316a" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 43723 } } peers { permanent_uuid: "a4bec2b8a6754c4eba7a45b19691e082" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 39543 } }
I20260926 06:24:08.418736 22425 ts_tablet_manager.cc:1428] T 65d70af6a1934a0486c0a45ab062e808 P b8602f6ef0814ad28fb9f1f2082c7fbd: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 06:24:08.418864 22277 heartbeater.cc:503] Master 127.4.202.190:42137 was elected leader, sending a full tablet report...
I20260926 06:24:08.418849 22429 log.cc:824] T 65d70af6a1934a0486c0a45ab062e808 P 82cec124bea34379b3741f49b809316a: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:08.418891 22426 raft_consensus.cc:348] T 65d70af6a1934a0486c0a45ab062e808 P a4bec2b8a6754c4eba7a45b19691e082 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8602f6ef0814ad28fb9f1f2082c7fbd" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 41957 } } peers { permanent_uuid: "82cec124bea34379b3741f49b809316a" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 43723 } } peers { permanent_uuid: "a4bec2b8a6754c4eba7a45b19691e082" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 39543 } }
I20260926 06:24:08.419181 22426 raft_consensus.cc:374] T 65d70af6a1934a0486c0a45ab062e808 P a4bec2b8a6754c4eba7a45b19691e082 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:08.419284 22426 raft_consensus.cc:729] T 65d70af6a1934a0486c0a45ab062e808 P a4bec2b8a6754c4eba7a45b19691e082 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4bec2b8a6754c4eba7a45b19691e082, State: Initialized, Role: FOLLOWER
I20260926 06:24:08.419414 22426 consensus_queue.cc:260] T 65d70af6a1934a0486c0a45ab062e808 P a4bec2b8a6754c4eba7a45b19691e082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8602f6ef0814ad28fb9f1f2082c7fbd" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 41957 } } peers { permanent_uuid: "82cec124bea34379b3741f49b809316a" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 43723 } } peers { permanent_uuid: "a4bec2b8a6754c4eba7a45b19691e082" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 39543 } }
I20260926 06:24:08.419574 22429 tablet_bootstrap.cc:492] T 65d70af6a1934a0486c0a45ab062e808 P 82cec124bea34379b3741f49b809316a: No bootstrap required, opened a new log
I20260926 06:24:08.419689 22429 ts_tablet_manager.cc:1397] T 65d70af6a1934a0486c0a45ab062e808 P 82cec124bea34379b3741f49b809316a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260926 06:24:08.419695 22143 heartbeater.cc:503] Master 127.4.202.190:42137 was elected leader, sending a full tablet report...
I20260926 06:24:08.419687 22426 ts_tablet_manager.cc:1428] T 65d70af6a1934a0486c0a45ab062e808 P a4bec2b8a6754c4eba7a45b19691e082: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260926 06:24:08.421250 22429 raft_consensus.cc:348] T 65d70af6a1934a0486c0a45ab062e808 P 82cec124bea34379b3741f49b809316a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8602f6ef0814ad28fb9f1f2082c7fbd" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 41957 } } peers { permanent_uuid: "82cec124bea34379b3741f49b809316a" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 43723 } } peers { permanent_uuid: "a4bec2b8a6754c4eba7a45b19691e082" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 39543 } }
I20260926 06:24:08.421373 22429 raft_consensus.cc:374] T 65d70af6a1934a0486c0a45ab062e808 P 82cec124bea34379b3741f49b809316a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:08.421394 22429 raft_consensus.cc:729] T 65d70af6a1934a0486c0a45ab062e808 P 82cec124bea34379b3741f49b809316a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82cec124bea34379b3741f49b809316a, State: Initialized, Role: FOLLOWER
I20260926 06:24:08.421520 22429 consensus_queue.cc:260] T 65d70af6a1934a0486c0a45ab062e808 P 82cec124bea34379b3741f49b809316a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8602f6ef0814ad28fb9f1f2082c7fbd" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 41957 } } peers { permanent_uuid: "82cec124bea34379b3741f49b809316a" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 43723 } } peers { permanent_uuid: "a4bec2b8a6754c4eba7a45b19691e082" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 39543 } }
I20260926 06:24:08.421782 22429 ts_tablet_manager.cc:1428] T 65d70af6a1934a0486c0a45ab062e808 P 82cec124bea34379b3741f49b809316a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 06:24:08.421803 22411 heartbeater.cc:503] Master 127.4.202.190:42137 was elected leader, sending a full tablet report...
W20260926 06:24:08.490183 22278 tablet.cc:2369] T 65d70af6a1934a0486c0a45ab062e808 P b8602f6ef0814ad28fb9f1f2082c7fbd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 06:24:08.587246 22144 tablet.cc:2369] T 65d70af6a1934a0486c0a45ab062e808 P a4bec2b8a6754c4eba7a45b19691e082: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:24:08.630096 22430 raft_consensus.cc:482] T 65d70af6a1934a0486c0a45ab062e808 P b8602f6ef0814ad28fb9f1f2082c7fbd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:24:08.630240 22430 raft_consensus.cc:504] T 65d70af6a1934a0486c0a45ab062e808 P b8602f6ef0814ad28fb9f1f2082c7fbd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8602f6ef0814ad28fb9f1f2082c7fbd" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 41957 } } peers { permanent_uuid: "82cec124bea34379b3741f49b809316a" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 43723 } } peers { permanent_uuid: "a4bec2b8a6754c4eba7a45b19691e082" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 39543 } }
I20260926 06:24:08.630738 22430 leader_election.cc:288] T 65d70af6a1934a0486c0a45ab062e808 P b8602f6ef0814ad28fb9f1f2082c7fbd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 82cec124bea34379b3741f49b809316a (127.4.202.131:43723), a4bec2b8a6754c4eba7a45b19691e082 (127.4.202.129:39543)
W20260926 06:24:08.630911 22412 tablet.cc:2369] T 65d70af6a1934a0486c0a45ab062e808 P 82cec124bea34379b3741f49b809316a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:24:08.633858 22424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:08.631160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:36923 (local address 127.4.202.131:43723)
0926 06:24:08.631217 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:08.631219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:08.631226 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:24:08.631252 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:08.631253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:08.631262 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:08.631340 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:24:08.631519 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.632138 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.633155 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.633523 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.633603 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:08.633610 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:08.633759 (+ 149us) server_negotiation.cc:299] Negotiation successful
0926 06:24:08.633792 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 06:24:08.633858 22421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:08.630930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:33925 (local address 127.4.202.129:39543)
0926 06:24:08.631155 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:08.631157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:08.631284 (+ 127us) server_negotiation.cc:407] Connection header received
0926 06:24:08.631343 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:08.631344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:08.631354 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:08.631419 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:24:08.631572 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.632137 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.633157 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.633524 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.633604 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:08.633612 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:08.633761 (+ 149us) server_negotiation.cc:299] Negotiation successful
0926 06:24:08.633793 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200}
I20260926 06:24:08.633869 22279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:08.630862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:43723 (local address 127.4.202.130:36923)
0926 06:24:08.631023 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:08.631034 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:08.631083 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:08.631345 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:08.631347 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:08.631349 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:24:08.631416 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:08.631418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.632222 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.632223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:08.633055 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:08.633057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.633600 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.633601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:08.633680 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:08.633683 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:24:08.633753 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 06:24:08.633792 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":60}
I20260926 06:24:08.633873 22434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:08.630871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:39543 (local address 127.4.202.130:33925)
0926 06:24:08.631219 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:08.631226 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:08.631287 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:08.631423 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:08.631424 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:08.631425 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:24:08.631496 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:08.631502 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.632219 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.632220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:08.633053 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:08.633055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.633604 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.633605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:08.633675 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:08.633679 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:24:08.633742 (+ 63us) client_negotiation.cc:240] Negotiation successful
0926 06:24:08.633782 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":34,"thread_start_us":98,"threads_started":1}
I20260926 06:24:08.634239 22097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65d70af6a1934a0486c0a45ab062e808" candidate_uuid: "b8602f6ef0814ad28fb9f1f2082c7fbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4bec2b8a6754c4eba7a45b19691e082" is_pre_election: true
I20260926 06:24:08.634390 22097 raft_consensus.cc:2454] T 65d70af6a1934a0486c0a45ab062e808 P a4bec2b8a6754c4eba7a45b19691e082 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8602f6ef0814ad28fb9f1f2082c7fbd in term 0.
I20260926 06:24:08.634361 22365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65d70af6a1934a0486c0a45ab062e808" candidate_uuid: "b8602f6ef0814ad28fb9f1f2082c7fbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82cec124bea34379b3741f49b809316a" is_pre_election: true
I20260926 06:24:08.634507 22365 raft_consensus.cc:2454] T 65d70af6a1934a0486c0a45ab062e808 P 82cec124bea34379b3741f49b809316a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8602f6ef0814ad28fb9f1f2082c7fbd in term 0.
I20260926 06:24:08.634642 22166 leader_election.cc:302] T 65d70af6a1934a0486c0a45ab062e808 P b8602f6ef0814ad28fb9f1f2082c7fbd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4bec2b8a6754c4eba7a45b19691e082, b8602f6ef0814ad28fb9f1f2082c7fbd; no voters:
I20260926 06:24:08.634814 22430 raft_consensus.cc:2781] T 65d70af6a1934a0486c0a45ab062e808 P b8602f6ef0814ad28fb9f1f2082c7fbd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:24:08.634861 22430 raft_consensus.cc:482] T 65d70af6a1934a0486c0a45ab062e808 P b8602f6ef0814ad28fb9f1f2082c7fbd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:24:08.634899 22430 raft_consensus.cc:3037] T 65d70af6a1934a0486c0a45ab062e808 P b8602f6ef0814ad28fb9f1f2082c7fbd [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:08.635597 22430 raft_consensus.cc:504] T 65d70af6a1934a0486c0a45ab062e808 P b8602f6ef0814ad28fb9f1f2082c7fbd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8602f6ef0814ad28fb9f1f2082c7fbd" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 41957 } } peers { permanent_uuid: "82cec124bea34379b3741f49b809316a" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 43723 } } peers { permanent_uuid: "a4bec2b8a6754c4eba7a45b19691e082" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 39543 } }
I20260926 06:24:08.635731 22430 leader_election.cc:288] T 65d70af6a1934a0486c0a45ab062e808 P b8602f6ef0814ad28fb9f1f2082c7fbd [CANDIDATE]: Term 1 election: Requested vote from peers 82cec124bea34379b3741f49b809316a (127.4.202.131:43723), a4bec2b8a6754c4eba7a45b19691e082 (127.4.202.129:39543)
I20260926 06:24:08.635902 22365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65d70af6a1934a0486c0a45ab062e808" candidate_uuid: "b8602f6ef0814ad28fb9f1f2082c7fbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82cec124bea34379b3741f49b809316a"
I20260926 06:24:08.635924 22097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65d70af6a1934a0486c0a45ab062e808" candidate_uuid: "b8602f6ef0814ad28fb9f1f2082c7fbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4bec2b8a6754c4eba7a45b19691e082"
I20260926 06:24:08.635967 22365 raft_consensus.cc:3037] T 65d70af6a1934a0486c0a45ab062e808 P 82cec124bea34379b3741f49b809316a [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:08.635995 22097 raft_consensus.cc:3037] T 65d70af6a1934a0486c0a45ab062e808 P a4bec2b8a6754c4eba7a45b19691e082 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:08.636673 22097 raft_consensus.cc:2454] T 65d70af6a1934a0486c0a45ab062e808 P a4bec2b8a6754c4eba7a45b19691e082 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8602f6ef0814ad28fb9f1f2082c7fbd in term 1.
I20260926 06:24:08.636684 22365 raft_consensus.cc:2454] T 65d70af6a1934a0486c0a45ab062e808 P 82cec124bea34379b3741f49b809316a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8602f6ef0814ad28fb9f1f2082c7fbd in term 1.
I20260926 06:24:08.636848 22166 leader_election.cc:302] T 65d70af6a1934a0486c0a45ab062e808 P b8602f6ef0814ad28fb9f1f2082c7fbd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4bec2b8a6754c4eba7a45b19691e082, b8602f6ef0814ad28fb9f1f2082c7fbd; no voters:
I20260926 06:24:08.636948 22430 raft_consensus.cc:2781] T 65d70af6a1934a0486c0a45ab062e808 P b8602f6ef0814ad28fb9f1f2082c7fbd [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:24:08.637095 22430 raft_consensus.cc:686] T 65d70af6a1934a0486c0a45ab062e808 P b8602f6ef0814ad28fb9f1f2082c7fbd [term 1 LEADER]: Becoming Leader. State: Replica: b8602f6ef0814ad28fb9f1f2082c7fbd, State: Running, Role: LEADER
I20260926 06:24:08.637202 22430 consensus_queue.cc:237] T 65d70af6a1934a0486c0a45ab062e808 P b8602f6ef0814ad28fb9f1f2082c7fbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8602f6ef0814ad28fb9f1f2082c7fbd" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 41957 } } peers { permanent_uuid: "82cec124bea34379b3741f49b809316a" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 43723 } } peers { permanent_uuid: "a4bec2b8a6754c4eba7a45b19691e082" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 39543 } }
I20260926 06:24:08.637959 21952 catalog_manager.cc:5495] T 65d70af6a1934a0486c0a45ab062e808 P b8602f6ef0814ad28fb9f1f2082c7fbd reported cstate change: term changed from 0 to 1, leader changed from <none> to b8602f6ef0814ad28fb9f1f2082c7fbd (127.4.202.130). New cstate: current_term: 1 leader_uuid: "b8602f6ef0814ad28fb9f1f2082c7fbd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8602f6ef0814ad28fb9f1f2082c7fbd" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 41957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "82cec124bea34379b3741f49b809316a" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 43723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4bec2b8a6754c4eba7a45b19691e082" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 39543 } health_report { overall_health: UNKNOWN } } }
I20260926 06:24:08.690516 22423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:08.688140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46116 (local address 127.4.202.130:41957)
0926 06:24:08.688205 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:08.688206 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:08.688213 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:24:08.688243 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:08.688244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:08.688260 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:08.688334 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:24:08.688481 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.688954 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.689713 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.689927 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.689993 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:08.690164 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:24:08.690273 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:24:08.690361 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:08.690380 (+ 19us) server_negotiation.cc:299] Negotiation successful
0926 06:24:08.690400 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 06:24:08.692365 22365 raft_consensus.cc:1261] T 65d70af6a1934a0486c0a45ab062e808 P 82cec124bea34379b3741f49b809316a [term 1 FOLLOWER]: Refusing update from remote peer b8602f6ef0814ad28fb9f1f2082c7fbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:24:08.692680 22097 raft_consensus.cc:1261] T 65d70af6a1934a0486c0a45ab062e808 P a4bec2b8a6754c4eba7a45b19691e082 [term 1 FOLLOWER]: Refusing update from remote peer b8602f6ef0814ad28fb9f1f2082c7fbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:24:08.692736 22430 consensus_queue.cc:1035] T 65d70af6a1934a0486c0a45ab062e808 P b8602f6ef0814ad28fb9f1f2082c7fbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "82cec124bea34379b3741f49b809316a" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 43723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:24:08.692925 22430 consensus_queue.cc:1035] T 65d70af6a1934a0486c0a45ab062e808 P b8602f6ef0814ad28fb9f1f2082c7fbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4bec2b8a6754c4eba7a45b19691e082" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 39543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:24:08.695786 22439 mvcc.cc:205] Tried to move back new op lower bound from 7333494164240916480 to 7333494164018528256. Current Snapshot: MvccSnapshot[applied={T|T < 7333494164240916480}]
I20260926 06:24:08.698011 21952 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36774:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:24:08.698107 21952 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36774:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:24:08.698839 21952 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34c5dde05928457cb0ca7a3959b9923c: Sending DeleteTablet for 3 replicas of tablet 65d70af6a1934a0486c0a45ab062e808
I20260926 06:24:08.699132 22211 tablet_service.cc:1514] Processing DeleteTablet for tablet 65d70af6a1934a0486c0a45ab062e808 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:24:08 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:46100
I20260926 06:24:08.699157 22077 tablet_service.cc:1514] Processing DeleteTablet for tablet 65d70af6a1934a0486c0a45ab062e808 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:24:08 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:35308
I20260926 06:24:08.699247 22345 tablet_service.cc:1514] Processing DeleteTablet for tablet 65d70af6a1934a0486c0a45ab062e808 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:24:08 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:46288
I20260926 06:24:08.699378 22447 tablet_replica.cc:330] T 65d70af6a1934a0486c0a45ab062e808 P b8602f6ef0814ad28fb9f1f2082c7fbd: stopping tablet replica
I20260926 06:24:08.699484 22447 raft_consensus.cc:2229] T 65d70af6a1934a0486c0a45ab062e808 P b8602f6ef0814ad28fb9f1f2082c7fbd [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:24:08.699594 22447 raft_consensus.cc:2258] T 65d70af6a1934a0486c0a45ab062e808 P b8602f6ef0814ad28fb9f1f2082c7fbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:24:08.699687 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 22014
I20260926 06:24:08.703768 22447 ts_tablet_manager.cc:1905] T 65d70af6a1934a0486c0a45ab062e808 P b8602f6ef0814ad28fb9f1f2082c7fbd: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 06:24:08.704638 21938 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.202.129:39543 (error 108)
I20260926 06:24:08.704883 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 22148
I20260926 06:24:08.705413 22449 tablet_replica.cc:330] T 65d70af6a1934a0486c0a45ab062e808 P 82cec124bea34379b3741f49b809316a: stopping tablet replica
I20260926 06:24:08.705523 22449 raft_consensus.cc:2229] T 65d70af6a1934a0486c0a45ab062e808 P 82cec124bea34379b3741f49b809316a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:24:08.705618 22449 raft_consensus.cc:2258] T 65d70af6a1934a0486c0a45ab062e808 P 82cec124bea34379b3741f49b809316a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:24:08.705724 22419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:08.705604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:39543 (local address 127.0.0.1:35322)
0926 06:24:08.705650 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:08.705684 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.129:39543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260926 06:24:08.705821 21938 catalog_manager.cc:4584] TS a4bec2b8a6754c4eba7a45b19691e082 (127.4.202.129:39543): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 65d70af6a1934a0486c0a45ab062e808: Network error: Client connection negotiation failed: client connection to 127.4.202.129:39543: connect: Connection refused (error 111)
I20260926 06:24:08.705994 22449 ts_tablet_manager.cc:1905] T 65d70af6a1934a0486c0a45ab062e808 P 82cec124bea34379b3741f49b809316a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:24:08.707871 22449 ts_tablet_manager.cc:1918] T 65d70af6a1934a0486c0a45ab062e808 P 82cec124bea34379b3741f49b809316a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:24:08.707943 22449 log.cc:1196] T 65d70af6a1934a0486c0a45ab062e808 P 82cec124bea34379b3741f49b809316a: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790403591640726-4906-0/minicluster-data/ts-2/wal/wals/65d70af6a1934a0486c0a45ab062e808
I20260926 06:24:08.708179 22449 ts_tablet_manager.cc:1939] T 65d70af6a1934a0486c0a45ab062e808 P 82cec124bea34379b3741f49b809316a: Deleting consensus metadata
I20260926 06:24:08.708585 21936 catalog_manager.cc:4841] TS 82cec124bea34379b3741f49b809316a (127.4.202.131:43723): tablet 65d70af6a1934a0486c0a45ab062e808 (table test-table [id=fa906790a297428c98ea2585f0d24d1d]) successfully deleted
I20260926 06:24:08.710234 22419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:08.710103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:41957 (local address 127.0.0.1:46126)
0926 06:24:08.710155 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:08.710186 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.130:41957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 06:24:08.710485 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 22282
I20260926 06:24:08.716069 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 21920
2026-09-26T06:24:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2076 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-26T06:24:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:24:08Z Disabled control of system clock
I20260926 06:24:08.743204 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:36605
--webserver_interface=127.4.202.190
--webserver_port=0
--builtin_ntp_servers=127.4.202.148:33885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:36605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 06:24:08.821549 22454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:08.821744 22454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:08.821764 22454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:08.822832 22454 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:24:08.822885 22454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:08.822897 22454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:08.822909 22454 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:24:08.822921 22454 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:24:08.824169 22454 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:33885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:36605
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:36605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:08.824337 22454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:08.824549 22454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:08.829175 22462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:08.829186 22454 server_base.cc:1034] running on GCE node
W20260926 06:24:08.829159 22459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:08.829150 22460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:08.829669 22454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:08.829972 22454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:08.831126 22454 hybrid_clock.cc:648] HybridClock initialized: now 1790403848831113 us; error 24 us; skew 500 ppm
I20260926 06:24:08.832306 22454 webserver.cc:460] Webserver started at http://127.4.202.190:41209/ using document root <none> and password file <none>
I20260926 06:24:08.832547 22454 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:08.832613 22454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:08.832741 22454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:08.833698 22454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "081cd220eecc4d7980e5710a0ef7996e"
format_stamp: "Formatted at 2026-09-26 06:24:08 on dist-test-slave-pd0t"
server_key: "f82470387bcd54e639f32a4ae21d4a56"
server_key_iv: "b15c5a8efc0af09f2b284a6244b4c13c"
server_key_version: "encryptionkey@0"
I20260926 06:24:08.834028 22454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "081cd220eecc4d7980e5710a0ef7996e"
format_stamp: "Formatted at 2026-09-26 06:24:08 on dist-test-slave-pd0t"
server_key: "f82470387bcd54e639f32a4ae21d4a56"
server_key_iv: "b15c5a8efc0af09f2b284a6244b4c13c"
server_key_version: "encryptionkey@0"
I20260926 06:24:08.835354 22454 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:24:08.836006 22468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:08.836165 22454 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:24:08.836254 22454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "081cd220eecc4d7980e5710a0ef7996e"
format_stamp: "Formatted at 2026-09-26 06:24:08 on dist-test-slave-pd0t"
server_key: "f82470387bcd54e639f32a4ae21d4a56"
server_key_iv: "b15c5a8efc0af09f2b284a6244b4c13c"
server_key_version: "encryptionkey@0"
I20260926 06:24:08.836326 22454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:08.844370 22454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:08.844620 22454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:08.844753 22454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:08.848589 22454 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:36605
I20260926 06:24:08.848618 22520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:36605 every 8 connection(s)
I20260926 06:24:08.848903 22454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:24:08.849759 22521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:08.851788 22521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 081cd220eecc4d7980e5710a0ef7996e: Bootstrap starting.
I20260926 06:24:08.852267 22521 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 081cd220eecc4d7980e5710a0ef7996e: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:08.852486 22521 log.cc:824] T 00000000000000000000000000000000 P 081cd220eecc4d7980e5710a0ef7996e: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:08.853062 22521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 081cd220eecc4d7980e5710a0ef7996e: No bootstrap required, opened a new log
I20260926 06:24:08.854477 22521 raft_consensus.cc:348] T 00000000000000000000000000000000 P 081cd220eecc4d7980e5710a0ef7996e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "081cd220eecc4d7980e5710a0ef7996e" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 36605 } }
I20260926 06:24:08.854643 22521 raft_consensus.cc:374] T 00000000000000000000000000000000 P 081cd220eecc4d7980e5710a0ef7996e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:08.854687 22521 raft_consensus.cc:729] T 00000000000000000000000000000000 P 081cd220eecc4d7980e5710a0ef7996e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 081cd220eecc4d7980e5710a0ef7996e, State: Initialized, Role: FOLLOWER
I20260926 06:24:08.854780 22521 consensus_queue.cc:260] T 00000000000000000000000000000000 P 081cd220eecc4d7980e5710a0ef7996e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "081cd220eecc4d7980e5710a0ef7996e" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 36605 } }
I20260926 06:24:08.854831 22521 raft_consensus.cc:388] T 00000000000000000000000000000000 P 081cd220eecc4d7980e5710a0ef7996e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:24:08.854851 22521 raft_consensus.cc:482] T 00000000000000000000000000000000 P 081cd220eecc4d7980e5710a0ef7996e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:24:08.854875 22521 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 081cd220eecc4d7980e5710a0ef7996e [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:08.855314 22521 raft_consensus.cc:504] T 00000000000000000000000000000000 P 081cd220eecc4d7980e5710a0ef7996e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "081cd220eecc4d7980e5710a0ef7996e" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 36605 } }
I20260926 06:24:08.855391 22521 leader_election.cc:302] T 00000000000000000000000000000000 P 081cd220eecc4d7980e5710a0ef7996e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 081cd220eecc4d7980e5710a0ef7996e; no voters:
I20260926 06:24:08.855501 22521 leader_election.cc:288] T 00000000000000000000000000000000 P 081cd220eecc4d7980e5710a0ef7996e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:24:08.855571 22524 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 081cd220eecc4d7980e5710a0ef7996e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:24:08.855727 22524 raft_consensus.cc:686] T 00000000000000000000000000000000 P 081cd220eecc4d7980e5710a0ef7996e [term 1 LEADER]: Becoming Leader. State: Replica: 081cd220eecc4d7980e5710a0ef7996e, State: Running, Role: LEADER
I20260926 06:24:08.855738 22521 sys_catalog.cc:564] T 00000000000000000000000000000000 P 081cd220eecc4d7980e5710a0ef7996e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:24:08.855831 22524 consensus_queue.cc:237] T 00000000000000000000000000000000 P 081cd220eecc4d7980e5710a0ef7996e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "081cd220eecc4d7980e5710a0ef7996e" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 36605 } }
I20260926 06:24:08.856238 22526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 081cd220eecc4d7980e5710a0ef7996e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 081cd220eecc4d7980e5710a0ef7996e. Latest consensus state: current_term: 1 leader_uuid: "081cd220eecc4d7980e5710a0ef7996e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "081cd220eecc4d7980e5710a0ef7996e" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 36605 } } }
I20260926 06:24:08.856271 22525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 081cd220eecc4d7980e5710a0ef7996e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "081cd220eecc4d7980e5710a0ef7996e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "081cd220eecc4d7980e5710a0ef7996e" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 36605 } } }
I20260926 06:24:08.856452 22526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 081cd220eecc4d7980e5710a0ef7996e [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:08.856727 22525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 081cd220eecc4d7980e5710a0ef7996e [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:08.856897 22536 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:24:08.857046 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 22454
I20260926 06:24:08.857160 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:24:08.857368 4906 external_mini_cluster.cc:1428] Setting key d20e5a1251e77ecc13d90060c837607c
I20260926 06:24:08.857385 22536 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:24:08.858918 22536 catalog_manager.cc:1347] Generated new cluster ID: e327d15a66d8487b9f4d944ea2bde310
I20260926 06:24:08.858975 22536 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:24:08.863168 22543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:08.857776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56852 (local address 127.4.202.190:36605)
0926 06:24:08.857941 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:08.857949 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:08.858132 (+ 183us) server_negotiation.cc:407] Connection header received
0926 06:24:08.858239 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:08.858246 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:08.858310 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:08.858473 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:08.858699 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.859985 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.861081 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:08.861707 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:08.861829 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:08.862459 (+ 630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:08.862463 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:08.862463 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:08.862680 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:08.862719 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:08.862735 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:08.862803 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:08.862871 (+ 68us) server_negotiation.cc:299] Negotiation successful
0926 06:24:08.862946 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260926 06:24:08.868800 22536 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:24:08.869293 22536 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:24:08.891379 22536 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 081cd220eecc4d7980e5710a0ef7996e: Generated new TSK 0
I20260926 06:24:08.891543 22536 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:24:08.900502 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.129:0
--local_ip_for_outbound_sockets=127.4.202.129
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:36605
--builtin_ntp_servers=127.4.202.148:33885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 06:24:08.979590 22545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:08.979773 22545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:08.979795 22545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:08.980921 22545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:08.980963 22545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:08.980988 22545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.129
I20260926 06:24:08.982393 22545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:33885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:36605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.129
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:08.982669 22545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:08.982975 22545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:08.987674 22550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:08.987689 22551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:08.987690 22553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:08.987903 22545 server_base.cc:1034] running on GCE node
I20260926 06:24:08.988299 22545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:08.988569 22545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:08.989723 22545 hybrid_clock.cc:648] HybridClock initialized: now 1790403848989717 us; error 33 us; skew 500 ppm
I20260926 06:24:08.990885 22545 webserver.cc:460] Webserver started at http://127.4.202.129:41393/ using document root <none> and password file <none>
I20260926 06:24:08.991114 22545 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:08.991180 22545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:08.991326 22545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:08.992316 22545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-0/data/instance:
uuid: "1e4d25b765984d719c55b6d7d9f4da23"
format_stamp: "Formatted at 2026-09-26 06:24:08 on dist-test-slave-pd0t"
server_key: "a47aae0c47cfe97da1dc0055c7dc63e4"
server_key_iv: "5c024d81c5d003cd7093d85e10d6e4cf"
server_key_version: "encryptionkey@0"
I20260926 06:24:08.992671 22545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance:
uuid: "1e4d25b765984d719c55b6d7d9f4da23"
format_stamp: "Formatted at 2026-09-26 06:24:08 on dist-test-slave-pd0t"
server_key: "a47aae0c47cfe97da1dc0055c7dc63e4"
server_key_iv: "5c024d81c5d003cd7093d85e10d6e4cf"
server_key_version: "encryptionkey@0"
I20260926 06:24:08.993948 22545 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:24:08.994769 22559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:08.994942 22545 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:24:08.995045 22545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-0/wal
uuid: "1e4d25b765984d719c55b6d7d9f4da23"
format_stamp: "Formatted at 2026-09-26 06:24:08 on dist-test-slave-pd0t"
server_key: "a47aae0c47cfe97da1dc0055c7dc63e4"
server_key_iv: "5c024d81c5d003cd7093d85e10d6e4cf"
server_key_version: "encryptionkey@0"
I20260926 06:24:08.995127 22545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:09.009374 22545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:09.009644 22545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:09.009773 22545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:09.009972 22545 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:24:09.010257 22545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:24:09.010305 22545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:09.010342 22545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:24:09.010370 22545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:09.016177 22545 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.129:43109
I20260926 06:24:09.016232 22672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.129:43109 every 8 connection(s)
I20260926 06:24:09.016544 22545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
I20260926 06:24:09.021422 22543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:09.017252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:57409 (local address 127.4.202.190:36605)
0926 06:24:09.017296 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:09.017298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:09.017551 (+ 253us) server_negotiation.cc:407] Connection header received
0926 06:24:09.018029 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:09.018030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:09.018040 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:09.018080 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:09.018815 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.019306 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.020228 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.020409 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.020471 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:09.021218 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:09.021219 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:09.021219 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:09.021222 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:09.021232 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:09.021235 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:09.021272 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:09.021344 (+ 72us) server_negotiation.cc:299] Negotiation successful
0926 06:24:09.021366 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 06:24:09.021615 22675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:09.017232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:36605 (local address 127.4.202.129:57409)
0926 06:24:09.017423 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:09.017463 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:09.017936 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:09.018134 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:09.018137 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:09.018145 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:09.018715 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:09.018721 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.019396 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.019397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:09.020155 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:09.020157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.020479 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.020480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:09.020578 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:09.021104 (+ 526us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:24:09.021105 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:09.021131 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:24:09.021152 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:09.021284 (+ 132us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:09.021289 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:09.021290 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 06:24:09.021339 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 06:24:09.021411 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":91,"threads_started":1}
I20260926 06:24:09.022085 22673 heartbeater.cc:347] Connected to a master server at 127.4.202.190:36605
I20260926 06:24:09.022179 22673 heartbeater.cc:464] Registering TS with master...
I20260926 06:24:09.022305 22673 heartbeater.cc:511] Master 127.4.202.190:36605 requested a full tablet report, sending...
I20260926 06:24:09.022764 22485 ts_manager.cc:194] Registered new tserver with Master: 1e4d25b765984d719c55b6d7d9f4da23 (127.4.202.129:43109)
I20260926 06:24:09.023365 22485 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.202.129:57409
I20260926 06:24:09.025188 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 22545
I20260926 06:24:09.025296 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance
I20260926 06:24:09.025482 4906 external_mini_cluster.cc:1428] Setting key 8e5084266de5c3578bf62a7fedf649ce
I20260926 06:24:09.026314 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.130:0
--local_ip_for_outbound_sockets=127.4.202.130
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:36605
--builtin_ntp_servers=127.4.202.148:33885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 06:24:09.105757 22676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:09.105997 22676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:09.106032 22676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:09.107226 22676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:09.107282 22676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:09.107319 22676 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.130
I20260926 06:24:09.108605 22676 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:33885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:36605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.130
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:09.108848 22676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:09.109113 22676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:09.113862 22684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:09.113958 22682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:09.114001 22681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:09.114193 22676 server_base.cc:1034] running on GCE node
I20260926 06:24:09.114401 22676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:09.114665 22676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:09.115815 22676 hybrid_clock.cc:648] HybridClock initialized: now 1790403849115804 us; error 24 us; skew 500 ppm
I20260926 06:24:09.116978 22676 webserver.cc:460] Webserver started at http://127.4.202.130:34517/ using document root <none> and password file <none>
I20260926 06:24:09.117208 22676 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:09.117269 22676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:09.117398 22676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:09.118458 22676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-1/data/instance:
uuid: "7728023e1d924143becbd113528dadec"
format_stamp: "Formatted at 2026-09-26 06:24:09 on dist-test-slave-pd0t"
server_key: "68e44b1acc9e1ffd50c3756e6b54a8b2"
server_key_iv: "3de2ba43ff97815976b9258f585069f1"
server_key_version: "encryptionkey@0"
I20260926 06:24:09.118849 22676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance:
uuid: "7728023e1d924143becbd113528dadec"
format_stamp: "Formatted at 2026-09-26 06:24:09 on dist-test-slave-pd0t"
server_key: "68e44b1acc9e1ffd50c3756e6b54a8b2"
server_key_iv: "3de2ba43ff97815976b9258f585069f1"
server_key_version: "encryptionkey@0"
I20260926 06:24:09.120113 22676 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:24:09.120836 22690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:09.121019 22676 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:24:09.121112 22676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-1/wal
uuid: "7728023e1d924143becbd113528dadec"
format_stamp: "Formatted at 2026-09-26 06:24:09 on dist-test-slave-pd0t"
server_key: "68e44b1acc9e1ffd50c3756e6b54a8b2"
server_key_iv: "3de2ba43ff97815976b9258f585069f1"
server_key_version: "encryptionkey@0"
I20260926 06:24:09.121191 22676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:09.149436 22676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:09.149752 22676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:09.149889 22676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:09.150163 22676 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:24:09.150486 22676 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:24:09.150538 22676 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:09.150600 22676 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:24:09.150629 22676 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:09.156147 22676 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.130:44149
I20260926 06:24:09.156261 22803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.130:44149 every 8 connection(s)
I20260926 06:24:09.156585 22676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
I20260926 06:24:09.160794 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 22676
I20260926 06:24:09.160894 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance
I20260926 06:24:09.161239 4906 external_mini_cluster.cc:1428] Setting key 42ce6130e6b435d77ae95f44417e8298
I20260926 06:24:09.161836 22543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:09.157323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:38631 (local address 127.4.202.190:36605)
0926 06:24:09.157384 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:09.157386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:09.157596 (+ 210us) server_negotiation.cc:407] Connection header received
0926 06:24:09.158105 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:09.158106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:09.158116 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:09.158173 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:09.158895 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.159399 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.160334 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.160514 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.160564 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:09.161588 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:09.161589 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:09.161589 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:09.161594 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:09.161609 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:09.161613 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:09.161666 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:09.161743 (+ 77us) server_negotiation.cc:299] Negotiation successful
0926 06:24:09.161766 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 06:24:09.162045 22806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:09.157321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:36605 (local address 127.4.202.130:38631)
0926 06:24:09.157489 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:09.157520 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:09.157990 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:09.158224 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:09.158227 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:09.158235 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:09.158784 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:09.158790 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.159484 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.159485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:09.160259 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:09.160260 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.160627 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.160628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:09.160783 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:09.161434 (+ 651us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:24:09.161435 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:09.161463 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:24:09.161485 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:09.161668 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:09.161672 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:09.161675 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:24:09.161733 (+ 58us) client_negotiation.cc:240] Negotiation successful
0926 06:24:09.161808 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":84,"threads_started":1}
I20260926 06:24:09.162358 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.131:0
--local_ip_for_outbound_sockets=127.4.202.131
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:36605
--builtin_ntp_servers=127.4.202.148:33885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 06:24:09.162528 22804 heartbeater.cc:347] Connected to a master server at 127.4.202.190:36605
I20260926 06:24:09.162629 22804 heartbeater.cc:464] Registering TS with master...
I20260926 06:24:09.162794 22804 heartbeater.cc:511] Master 127.4.202.190:36605 requested a full tablet report, sending...
I20260926 06:24:09.163159 22485 ts_manager.cc:194] Registered new tserver with Master: 7728023e1d924143becbd113528dadec (127.4.202.130:44149)
I20260926 06:24:09.163548 22485 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.202.130:38631
W20260926 06:24:09.240993 22807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:09.241171 22807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:09.241192 22807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:09.242311 22807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:09.242350 22807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:09.242375 22807 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.131
I20260926 06:24:09.243743 22807 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:33885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:36605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.131
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:09.243973 22807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:09.244252 22807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:09.249086 22812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:09.249094 22813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:09.249094 22815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:09.249325 22807 server_base.cc:1034] running on GCE node
I20260926 06:24:09.249552 22807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:09.249775 22807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:09.250936 22807 hybrid_clock.cc:648] HybridClock initialized: now 1790403849250918 us; error 43 us; skew 500 ppm
I20260926 06:24:09.252147 22807 webserver.cc:460] Webserver started at http://127.4.202.131:34051/ using document root <none> and password file <none>
I20260926 06:24:09.252383 22807 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:09.252462 22807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:09.252604 22807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:09.253597 22807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-2/data/instance:
uuid: "24e8b3b398de4936b4418f0577b6822b"
format_stamp: "Formatted at 2026-09-26 06:24:09 on dist-test-slave-pd0t"
server_key: "243ebd562305e30a6d22db0628e63c73"
server_key_iv: "dc0a0997a14f11b38822052eed549a95"
server_key_version: "encryptionkey@0"
I20260926 06:24:09.253928 22807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance:
uuid: "24e8b3b398de4936b4418f0577b6822b"
format_stamp: "Formatted at 2026-09-26 06:24:09 on dist-test-slave-pd0t"
server_key: "243ebd562305e30a6d22db0628e63c73"
server_key_iv: "dc0a0997a14f11b38822052eed549a95"
server_key_version: "encryptionkey@0"
I20260926 06:24:09.255241 22807 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:24:09.255949 22821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:09.256117 22807 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:24:09.256207 22807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-2/wal
uuid: "24e8b3b398de4936b4418f0577b6822b"
format_stamp: "Formatted at 2026-09-26 06:24:09 on dist-test-slave-pd0t"
server_key: "243ebd562305e30a6d22db0628e63c73"
server_key_iv: "dc0a0997a14f11b38822052eed549a95"
server_key_version: "encryptionkey@0"
I20260926 06:24:09.256278 22807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:09.277359 22807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:09.277633 22807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:09.277776 22807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:09.277979 22807 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:24:09.278266 22807 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:24:09.278316 22807 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:09.278353 22807 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:24:09.278381 22807 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:09.283809 22807 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.131:43215
I20260926 06:24:09.283856 22934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.131:43215 every 8 connection(s)
I20260926 06:24:09.284149 22807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
I20260926 06:24:09.287079 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 22807
I20260926 06:24:09.287199 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance
I20260926 06:24:09.287405 4906 external_mini_cluster.cc:1428] Setting key 0e14977c092fc9204708f12c02cc1659
I20260926 06:24:09.289556 22543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:09.284780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:47659 (local address 127.4.202.190:36605)
0926 06:24:09.284857 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:09.284859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:09.285063 (+ 204us) server_negotiation.cc:407] Connection header received
0926 06:24:09.285511 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:09.285512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:09.285522 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:09.285573 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:09.286182 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.286730 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.288172 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.288372 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.288446 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:09.289280 (+ 834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:09.289281 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:09.289282 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:09.289286 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:09.289301 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:09.289305 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:09.289372 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:09.289449 (+ 77us) server_negotiation.cc:299] Negotiation successful
0926 06:24:09.289481 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 06:24:09.289765 22937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:09.284750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:36605 (local address 127.4.202.131:47659)
0926 06:24:09.284940 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:09.284974 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:09.285419 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:09.285624 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:09.285627 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:09.285635 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:09.286113 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:09.286117 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.286805 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.286806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:09.288085 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:09.288087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.288452 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.288454 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:09.288597 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:09.289126 (+ 529us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:24:09.289126 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:09.289153 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:24:09.289175 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:09.289372 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:09.289378 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:09.289381 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:24:09.289442 (+ 61us) client_negotiation.cc:240] Negotiation successful
0926 06:24:09.289538 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":101,"threads_started":1}
I20260926 06:24:09.290228 22935 heartbeater.cc:347] Connected to a master server at 127.4.202.190:36605
I20260926 06:24:09.290302 22935 heartbeater.cc:464] Registering TS with master...
I20260926 06:24:09.290481 22935 heartbeater.cc:511] Master 127.4.202.190:36605 requested a full tablet report, sending...
I20260926 06:24:09.290845 22485 ts_manager.cc:194] Registered new tserver with Master: 24e8b3b398de4936b4418f0577b6822b (127.4.202.131:43215)
I20260926 06:24:09.291198 22485 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.202.131:47659
I20260926 06:24:09.299584 4906 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:24:09.303378 22543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:09.300905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56854 (local address 127.4.202.190:36605)
0926 06:24:09.300953 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:09.300955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:09.301062 (+ 107us) server_negotiation.cc:407] Connection header received
0926 06:24:09.301105 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:09.301105 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:09.301115 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:09.301162 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:09.301274 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.301728 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.302400 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.302640 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.302693 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:09.303219 (+ 526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:09.303220 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:09.303220 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:09.303224 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:09.303235 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:09.303239 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:09.303275 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:09.303308 (+ 33us) server_negotiation.cc:299] Negotiation successful
0926 06:24:09.303328 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 06:24:09.304680 22485 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56854:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:24:09.306173 22485 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:24:09.312263 22947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:09.308385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60010 (local address 127.4.202.131:43215)
0926 06:24:09.308544 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:09.308549 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:09.308864 (+ 315us) server_negotiation.cc:407] Connection header received
0926 06:24:09.308912 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:09.308914 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:09.308930 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:09.309004 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:24:09.309190 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.310230 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.311404 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.311868 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.311967 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:09.312054 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:09.312147 (+ 93us) server_negotiation.cc:299] Negotiation successful
0926 06:24:09.312182 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":73,"threads_started":1}
I20260926 06:24:09.312254 22944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:09.308202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:43215 (local address 127.0.0.1:60010)
0926 06:24:09.308779 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:09.308787 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:09.308855 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:09.309075 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:09.309076 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:09.309078 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:24:09.309116 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:09.309117 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.310304 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.310305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:09.311316 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:09.311318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.311969 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.311970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:09.312049 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:09.312062 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 06:24:09.312135 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 06:24:09.312182 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"mutex_wait_us":2,"thread_start_us":73,"threads_started":1}
I20260926 06:24:09.312625 22945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:09.308187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:43109 (local address 127.0.0.1:46750)
0926 06:24:09.308649 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:09.308658 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:09.308722 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:09.309109 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:09.309110 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:09.309111 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:24:09.309141 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:09.309142 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.309980 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.309982 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:09.311319 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:09.311320 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.312343 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.312344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:09.312408 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:09.312414 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 06:24:09.312474 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 06:24:09.312521 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"mutex_wait_us":37,"thread_start_us":93,"threads_started":1}
I20260926 06:24:09.312692 22946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:09.308269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46750 (local address 127.4.202.129:43109)
0926 06:24:09.308580 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:09.308585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:09.308919 (+ 334us) server_negotiation.cc:407] Connection header received
0926 06:24:09.308963 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:09.308965 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:09.308981 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:09.309045 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:24:09.309213 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.309898 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.311416 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.312155 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.312224 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:09.312304 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:09.312514 (+ 210us) server_negotiation.cc:299] Negotiation successful
0926 06:24:09.312561 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":81,"threads_started":1}
I20260926 06:24:09.312624 22943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:09.308186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:44149 (local address 127.0.0.1:43066)
0926 06:24:09.308444 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:09.308483 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:09.308550 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:09.308735 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:09.308738 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:09.308740 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:24:09.308821 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:09.308822 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.310162 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.310163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:09.311313 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:09.311316 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.312342 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.312343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:09.312408 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:09.312416 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 06:24:09.312474 (+ 58us) client_negotiation.cc:240] Negotiation successful
0926 06:24:09.312521 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":10,"thread_start_us":49,"threads_started":1}
I20260926 06:24:09.313277 22869 tablet_service.cc:1467] Processing CreateTablet for tablet 0d1d10f1e5bb4b69a673f47f7f4ff84e (DEFAULT_TABLE table=test-table [id=f0ca201512694ade8c22d73f3859642f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:24:09.313278 22607 tablet_service.cc:1467] Processing CreateTablet for tablet 0d1d10f1e5bb4b69a673f47f7f4ff84e (DEFAULT_TABLE table=test-table [id=f0ca201512694ade8c22d73f3859642f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:24:09.313589 22869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d1d10f1e5bb4b69a673f47f7f4ff84e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:09.313606 22607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d1d10f1e5bb4b69a673f47f7f4ff84e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:09.314826 22948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:09.308397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43066 (local address 127.4.202.130:44149)
0926 06:24:09.308572 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:09.308575 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:09.308584 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:24:09.308624 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:09.308626 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:09.308644 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:09.308711 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:24:09.308926 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.310085 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.311415 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.312154 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.312224 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:09.312304 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:09.314700 (+ 2396us) server_negotiation.cc:299] Negotiation successful
0926 06:24:09.314744 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":81,"threads_started":1}
I20260926 06:24:09.315644 22738 tablet_service.cc:1467] Processing CreateTablet for tablet 0d1d10f1e5bb4b69a673f47f7f4ff84e (DEFAULT_TABLE table=test-table [id=f0ca201512694ade8c22d73f3859642f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:24:09.315953 22738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d1d10f1e5bb4b69a673f47f7f4ff84e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:09.316577 22949 tablet_bootstrap.cc:492] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 1e4d25b765984d719c55b6d7d9f4da23: Bootstrap starting.
I20260926 06:24:09.317121 22949 tablet_bootstrap.cc:654] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 1e4d25b765984d719c55b6d7d9f4da23: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:09.317409 22949 log.cc:824] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 1e4d25b765984d719c55b6d7d9f4da23: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:09.317878 22950 tablet_bootstrap.cc:492] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 24e8b3b398de4936b4418f0577b6822b: Bootstrap starting.
I20260926 06:24:09.318243 22949 tablet_bootstrap.cc:492] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 1e4d25b765984d719c55b6d7d9f4da23: No bootstrap required, opened a new log
I20260926 06:24:09.318316 22949 ts_tablet_manager.cc:1397] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 1e4d25b765984d719c55b6d7d9f4da23: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:24:09.318779 22952 tablet_bootstrap.cc:492] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 7728023e1d924143becbd113528dadec: Bootstrap starting.
I20260926 06:24:09.319314 22952 tablet_bootstrap.cc:654] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 7728023e1d924143becbd113528dadec: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:09.319577 22952 log.cc:824] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 7728023e1d924143becbd113528dadec: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:09.320170 22952 tablet_bootstrap.cc:492] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 7728023e1d924143becbd113528dadec: No bootstrap required, opened a new log
I20260926 06:24:09.320220 22952 ts_tablet_manager.cc:1397] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 7728023e1d924143becbd113528dadec: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 06:24:09.320288 22949 raft_consensus.cc:348] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 1e4d25b765984d719c55b6d7d9f4da23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7728023e1d924143becbd113528dadec" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44149 } } peers { permanent_uuid: "1e4d25b765984d719c55b6d7d9f4da23" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 43109 } } peers { permanent_uuid: "24e8b3b398de4936b4418f0577b6822b" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 43215 } }
I20260926 06:24:09.320426 22949 raft_consensus.cc:374] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 1e4d25b765984d719c55b6d7d9f4da23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:09.320469 22949 raft_consensus.cc:729] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 1e4d25b765984d719c55b6d7d9f4da23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e4d25b765984d719c55b6d7d9f4da23, State: Initialized, Role: FOLLOWER
I20260926 06:24:09.320586 22949 consensus_queue.cc:260] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 1e4d25b765984d719c55b6d7d9f4da23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7728023e1d924143becbd113528dadec" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44149 } } peers { permanent_uuid: "1e4d25b765984d719c55b6d7d9f4da23" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 43109 } } peers { permanent_uuid: "24e8b3b398de4936b4418f0577b6822b" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 43215 } }
I20260926 06:24:09.320863 22949 ts_tablet_manager.cc:1428] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 1e4d25b765984d719c55b6d7d9f4da23: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 06:24:09.320977 22673 heartbeater.cc:503] Master 127.4.202.190:36605 was elected leader, sending a full tablet report...
I20260926 06:24:09.321627 22950 tablet_bootstrap.cc:654] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 24e8b3b398de4936b4418f0577b6822b: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:09.321957 22950 log.cc:824] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 24e8b3b398de4936b4418f0577b6822b: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:09.322667 22950 tablet_bootstrap.cc:492] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 24e8b3b398de4936b4418f0577b6822b: No bootstrap required, opened a new log
I20260926 06:24:09.322564 22952 raft_consensus.cc:348] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 7728023e1d924143becbd113528dadec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7728023e1d924143becbd113528dadec" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44149 } } peers { permanent_uuid: "1e4d25b765984d719c55b6d7d9f4da23" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 43109 } } peers { permanent_uuid: "24e8b3b398de4936b4418f0577b6822b" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 43215 } }
I20260926 06:24:09.322724 22952 raft_consensus.cc:374] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 7728023e1d924143becbd113528dadec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:09.322742 22950 ts_tablet_manager.cc:1397] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 24e8b3b398de4936b4418f0577b6822b: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260926 06:24:09.322755 22952 raft_consensus.cc:729] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 7728023e1d924143becbd113528dadec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7728023e1d924143becbd113528dadec, State: Initialized, Role: FOLLOWER
I20260926 06:24:09.322852 22952 consensus_queue.cc:260] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 7728023e1d924143becbd113528dadec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7728023e1d924143becbd113528dadec" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44149 } } peers { permanent_uuid: "1e4d25b765984d719c55b6d7d9f4da23" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 43109 } } peers { permanent_uuid: "24e8b3b398de4936b4418f0577b6822b" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 43215 } }
I20260926 06:24:09.323052 22952 ts_tablet_manager.cc:1428] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 7728023e1d924143becbd113528dadec: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 06:24:09.323127 22804 heartbeater.cc:503] Master 127.4.202.190:36605 was elected leader, sending a full tablet report...
I20260926 06:24:09.324888 22950 raft_consensus.cc:348] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 24e8b3b398de4936b4418f0577b6822b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7728023e1d924143becbd113528dadec" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44149 } } peers { permanent_uuid: "1e4d25b765984d719c55b6d7d9f4da23" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 43109 } } peers { permanent_uuid: "24e8b3b398de4936b4418f0577b6822b" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 43215 } }
I20260926 06:24:09.325016 22950 raft_consensus.cc:374] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 24e8b3b398de4936b4418f0577b6822b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:09.325073 22950 raft_consensus.cc:729] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 24e8b3b398de4936b4418f0577b6822b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24e8b3b398de4936b4418f0577b6822b, State: Initialized, Role: FOLLOWER
I20260926 06:24:09.325192 22950 consensus_queue.cc:260] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 24e8b3b398de4936b4418f0577b6822b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7728023e1d924143becbd113528dadec" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44149 } } peers { permanent_uuid: "1e4d25b765984d719c55b6d7d9f4da23" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 43109 } } peers { permanent_uuid: "24e8b3b398de4936b4418f0577b6822b" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 43215 } }
I20260926 06:24:09.325443 22950 ts_tablet_manager.cc:1428] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 24e8b3b398de4936b4418f0577b6822b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 06:24:09.325497 22935 heartbeater.cc:503] Master 127.4.202.190:36605 was elected leader, sending a full tablet report...
W20260926 06:24:09.407109 22805 tablet.cc:2369] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 7728023e1d924143becbd113528dadec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:24:09.454777 22956 raft_consensus.cc:482] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 7728023e1d924143becbd113528dadec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:24:09.454885 22956 raft_consensus.cc:504] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 7728023e1d924143becbd113528dadec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7728023e1d924143becbd113528dadec" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44149 } } peers { permanent_uuid: "1e4d25b765984d719c55b6d7d9f4da23" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 43109 } } peers { permanent_uuid: "24e8b3b398de4936b4418f0577b6822b" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 43215 } }
I20260926 06:24:09.455314 22956 leader_election.cc:288] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 7728023e1d924143becbd113528dadec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e4d25b765984d719c55b6d7d9f4da23 (127.4.202.129:43109), 24e8b3b398de4936b4418f0577b6822b (127.4.202.131:43215)
I20260926 06:24:09.458353 22947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:09.455542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:60689 (local address 127.4.202.131:43215)
0926 06:24:09.455602 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:09.455604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:09.455614 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:24:09.455645 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:09.455646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:09.455655 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:09.455719 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:24:09.455900 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.456530 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.457683 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.458034 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.458087 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:09.458112 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:09.458239 (+ 127us) server_negotiation.cc:299] Negotiation successful
0926 06:24:09.458282 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 06:24:09.458372 22946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:09.455553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:38529 (local address 127.4.202.129:43109)
0926 06:24:09.455594 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:09.455596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:09.455719 (+ 123us) server_negotiation.cc:407] Connection header received
0926 06:24:09.455783 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:09.455784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:09.455796 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:09.455850 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:24:09.456020 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.456524 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.457703 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.458044 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.458127 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:09.458135 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:09.458280 (+ 145us) server_negotiation.cc:299] Negotiation successful
0926 06:24:09.458314 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:24:09.458359 22806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:09.455466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:43215 (local address 127.4.202.130:60689)
0926 06:24:09.455518 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:09.455530 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:09.455587 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:09.455730 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:09.455731 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:09.455733 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:24:09.455829 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:09.455831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.456603 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.456604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:09.457606 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:09.457608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.458108 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.458109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:09.458176 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:09.458178 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:24:09.458229 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 06:24:09.458280 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":29}
I20260926 06:24:09.458377 22958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:09.455489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:43109 (local address 127.4.202.130:38529)
0926 06:24:09.455648 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:09.455658 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:09.455713 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:09.455855 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:09.455855 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:09.455857 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:24:09.455942 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:09.455943 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.456634 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.456635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:09.457609 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:09.457611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.458138 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.458139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:09.458197 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:09.458200 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:24:09.458267 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 06:24:09.458298 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":1,"thread_start_us":95,"threads_started":1}
I20260926 06:24:09.458732 22627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d1d10f1e5bb4b69a673f47f7f4ff84e" candidate_uuid: "7728023e1d924143becbd113528dadec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e4d25b765984d719c55b6d7d9f4da23" is_pre_election: true
I20260926 06:24:09.458734 22889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d1d10f1e5bb4b69a673f47f7f4ff84e" candidate_uuid: "7728023e1d924143becbd113528dadec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24e8b3b398de4936b4418f0577b6822b" is_pre_election: true
I20260926 06:24:09.458873 22889 raft_consensus.cc:2454] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 24e8b3b398de4936b4418f0577b6822b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7728023e1d924143becbd113528dadec in term 0.
I20260926 06:24:09.458873 22627 raft_consensus.cc:2454] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 1e4d25b765984d719c55b6d7d9f4da23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7728023e1d924143becbd113528dadec in term 0.
I20260926 06:24:09.459100 22691 leader_election.cc:302] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 7728023e1d924143becbd113528dadec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e4d25b765984d719c55b6d7d9f4da23, 7728023e1d924143becbd113528dadec; no voters:
I20260926 06:24:09.459264 22956 raft_consensus.cc:2781] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 7728023e1d924143becbd113528dadec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:24:09.459306 22956 raft_consensus.cc:482] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 7728023e1d924143becbd113528dadec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:24:09.459321 22956 raft_consensus.cc:3037] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 7728023e1d924143becbd113528dadec [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:09.459944 22956 raft_consensus.cc:504] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 7728023e1d924143becbd113528dadec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7728023e1d924143becbd113528dadec" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44149 } } peers { permanent_uuid: "1e4d25b765984d719c55b6d7d9f4da23" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 43109 } } peers { permanent_uuid: "24e8b3b398de4936b4418f0577b6822b" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 43215 } }
I20260926 06:24:09.460137 22956 leader_election.cc:288] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 7728023e1d924143becbd113528dadec [CANDIDATE]: Term 1 election: Requested vote from peers 1e4d25b765984d719c55b6d7d9f4da23 (127.4.202.129:43109), 24e8b3b398de4936b4418f0577b6822b (127.4.202.131:43215)
I20260926 06:24:09.460207 22627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d1d10f1e5bb4b69a673f47f7f4ff84e" candidate_uuid: "7728023e1d924143becbd113528dadec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e4d25b765984d719c55b6d7d9f4da23"
I20260926 06:24:09.460203 22889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d1d10f1e5bb4b69a673f47f7f4ff84e" candidate_uuid: "7728023e1d924143becbd113528dadec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24e8b3b398de4936b4418f0577b6822b"
I20260926 06:24:09.460282 22627 raft_consensus.cc:3037] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 1e4d25b765984d719c55b6d7d9f4da23 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:09.460315 22889 raft_consensus.cc:3037] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 24e8b3b398de4936b4418f0577b6822b [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:09.461030 22627 raft_consensus.cc:2454] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 1e4d25b765984d719c55b6d7d9f4da23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7728023e1d924143becbd113528dadec in term 1.
I20260926 06:24:09.461031 22889 raft_consensus.cc:2454] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 24e8b3b398de4936b4418f0577b6822b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7728023e1d924143becbd113528dadec in term 1.
I20260926 06:24:09.461172 22691 leader_election.cc:302] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 7728023e1d924143becbd113528dadec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e4d25b765984d719c55b6d7d9f4da23, 7728023e1d924143becbd113528dadec; no voters:
I20260926 06:24:09.461261 22956 raft_consensus.cc:2781] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 7728023e1d924143becbd113528dadec [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:24:09.461385 22956 raft_consensus.cc:686] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 7728023e1d924143becbd113528dadec [term 1 LEADER]: Becoming Leader. State: Replica: 7728023e1d924143becbd113528dadec, State: Running, Role: LEADER
I20260926 06:24:09.461501 22956 consensus_queue.cc:237] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 7728023e1d924143becbd113528dadec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7728023e1d924143becbd113528dadec" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44149 } } peers { permanent_uuid: "1e4d25b765984d719c55b6d7d9f4da23" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 43109 } } peers { permanent_uuid: "24e8b3b398de4936b4418f0577b6822b" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 43215 } }
I20260926 06:24:09.462244 22485 catalog_manager.cc:5495] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 7728023e1d924143becbd113528dadec reported cstate change: term changed from 0 to 1, leader changed from <none> to 7728023e1d924143becbd113528dadec (127.4.202.130). New cstate: current_term: 1 leader_uuid: "7728023e1d924143becbd113528dadec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7728023e1d924143becbd113528dadec" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 44149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e4d25b765984d719c55b6d7d9f4da23" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 43109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e8b3b398de4936b4418f0577b6822b" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 43215 } health_report { overall_health: UNKNOWN } } }
I20260926 06:24:09.492750 22948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:09.490560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43080 (local address 127.4.202.130:44149)
0926 06:24:09.490621 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:09.490623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:09.490632 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:24:09.490693 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:09.490694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:09.490710 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:09.490775 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:24:09.490890 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.491334 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.492108 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:09.492289 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:09.492349 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:09.492479 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:24:09.492567 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:24:09.492628 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:09.492660 (+ 32us) server_negotiation.cc:299] Negotiation successful
0926 06:24:09.492678 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 06:24:09.494426 22889 raft_consensus.cc:1261] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 24e8b3b398de4936b4418f0577b6822b [term 1 FOLLOWER]: Refusing update from remote peer 7728023e1d924143becbd113528dadec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:24:09.494439 22627 raft_consensus.cc:1261] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 1e4d25b765984d719c55b6d7d9f4da23 [term 1 FOLLOWER]: Refusing update from remote peer 7728023e1d924143becbd113528dadec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:24:09.494788 22959 consensus_queue.cc:1035] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 7728023e1d924143becbd113528dadec [LEADER]: Connected to new peer: Peer: permanent_uuid: "24e8b3b398de4936b4418f0577b6822b" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 43215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:24:09.494910 22956 consensus_queue.cc:1035] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 7728023e1d924143becbd113528dadec [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e4d25b765984d719c55b6d7d9f4da23" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 43109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:24:09.497782 22963 mvcc.cc:205] Tried to move back new op lower bound from 7333494167526596608 to 7333494167394934784. Current Snapshot: MvccSnapshot[applied={T|T < 7333494167526596608}]
I20260926 06:24:09.497562 22962 mvcc.cc:205] Tried to move back new op lower bound from 7333494167526596608 to 7333494167394934784. Current Snapshot: MvccSnapshot[applied={T|T < 7333494167526596608 or (T in {7333494167526596608})}]
I20260926 06:24:09.499221 22484 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:24:09.499315 22484 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:24:09.500051 22484 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 081cd220eecc4d7980e5710a0ef7996e: Sending DeleteTablet for 3 replicas of tablet 0d1d10f1e5bb4b69a673f47f7f4ff84e
I20260926 06:24:09.500360 22738 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d1d10f1e5bb4b69a673f47f7f4ff84e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:24:09 UTC) from {username='slave'} at 127.0.0.1:43066
I20260926 06:24:09.500401 22869 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d1d10f1e5bb4b69a673f47f7f4ff84e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:24:09 UTC) from {username='slave'} at 127.0.0.1:60010
I20260926 06:24:09.500401 22607 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d1d10f1e5bb4b69a673f47f7f4ff84e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:24:09 UTC) from {username='slave'} at 127.0.0.1:46750
I20260926 06:24:09.500622 22973 tablet_replica.cc:330] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 1e4d25b765984d719c55b6d7d9f4da23: stopping tablet replica
I20260926 06:24:09.500725 22973 raft_consensus.cc:2229] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 1e4d25b765984d719c55b6d7d9f4da23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:24:09.500814 22973 raft_consensus.cc:2258] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 1e4d25b765984d719c55b6d7d9f4da23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:24:09.500865 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 22545
I20260926 06:24:09.500962 22974 tablet_replica.cc:330] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 24e8b3b398de4936b4418f0577b6822b: stopping tablet replica
I20260926 06:24:09.501049 22974 raft_consensus.cc:2229] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 24e8b3b398de4936b4418f0577b6822b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:24:09.501144 22974 raft_consensus.cc:2258] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 24e8b3b398de4936b4418f0577b6822b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:24:09.501642 22974 ts_tablet_manager.cc:1905] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 24e8b3b398de4936b4418f0577b6822b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:24:09.503247 22974 ts_tablet_manager.cc:1918] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 24e8b3b398de4936b4418f0577b6822b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:24:09.503315 22974 log.cc:1196] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 24e8b3b398de4936b4418f0577b6822b: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790403591640726-4906-0/minicluster-data/ts-2/wal/wals/0d1d10f1e5bb4b69a673f47f7f4ff84e
I20260926 06:24:09.503571 22974 ts_tablet_manager.cc:1939] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 24e8b3b398de4936b4418f0577b6822b: Deleting consensus metadata
I20260926 06:24:09.503959 22470 catalog_manager.cc:4841] TS 24e8b3b398de4936b4418f0577b6822b (127.4.202.131:43215): tablet 0d1d10f1e5bb4b69a673f47f7f4ff84e (table test-table [id=f0ca201512694ade8c22d73f3859642f]) successfully deleted
W20260926 06:24:09.506366 22469 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.202.129:43109 (error 108)
I20260926 06:24:09.506690 22972 tablet_replica.cc:330] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 7728023e1d924143becbd113528dadec: stopping tablet replica
I20260926 06:24:09.506772 22972 raft_consensus.cc:2229] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 7728023e1d924143becbd113528dadec [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:24:09.506877 22972 raft_consensus.cc:2258] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 7728023e1d924143becbd113528dadec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:24:09.507002 22945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:09.506877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:43109 (local address 127.0.0.1:46758)
0926 06:24:09.506928 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:09.506966 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.129:43109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260926 06:24:09.507164 22469 catalog_manager.cc:4584] TS 1e4d25b765984d719c55b6d7d9f4da23 (127.4.202.129:43109): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0d1d10f1e5bb4b69a673f47f7f4ff84e: Network error: Client connection negotiation failed: client connection to 127.4.202.129:43109: connect: Connection refused (error 111)
I20260926 06:24:09.507270 22972 ts_tablet_manager.cc:1905] T 0d1d10f1e5bb4b69a673f47f7f4ff84e P 7728023e1d924143becbd113528dadec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:24:09.507357 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 22676
I20260926 06:24:09.512871 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 22807
I20260926 06:24:09.512972 22945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:09.512847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:44149 (local address 127.0.0.1:43090)
0926 06:24:09.512885 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:09.512928 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.130:44149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260926 06:24:09.517221 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 22454
2026-09-26T06:24:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (800 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:24:09 dist-test-slave-pd0t krb5kdc[22978](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:24:09 dist-test-slave-pd0t krb5kdc[22978](info): set up 2 sockets
Sep 26 06:24:09 dist-test-slave-pd0t krb5kdc[22978](info): commencing operation
krb5kdc: starting...
W20260926 06:24:10.550340 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:24:10 dist-test-slave-pd0t krb5kdc[22978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:24:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:24:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.202.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
I20260926 06:24:10.667601 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:40799
--webserver_interface=127.4.202.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:34467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:40799
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:24:10.746140 22991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:10.746400 22991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:10.746443 22991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:10.747737 22991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:24:10.747814 22991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:10.747840 22991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:10.747862 22991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:24:10.747897 22991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:24:10.749495 22991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:34467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:40799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:40799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:10.749818 22991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:10.750063 22991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:10.755194 22999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:10.755191 22997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:10.755187 22996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:10.755578 22991 server_base.cc:1034] running on GCE node
I20260926 06:24:10.755842 22991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:10.756132 22991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:10.757279 22991 hybrid_clock.cc:648] HybridClock initialized: now 1790403850757267 us; error 32 us; skew 500 ppm
Sep 26 06:24:10 dist-test-slave-pd0t krb5kdc[22978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403850, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:10.759781 22991 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:24:10.760277 22991 webserver.cc:460] Webserver started at http://127.4.202.190:45323/ using document root <none> and password file <none>
I20260926 06:24:10.760515 22991 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:10.760591 22991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:10.760746 22991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:10.761834 22991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "ef2f80cc690b4703aff5e73fb780295d"
format_stamp: "Formatted at 2026-09-26 06:24:10 on dist-test-slave-pd0t"
server_key: "2cd69a2c3b4ae08e52a4e2bc090f6b7d"
server_key_iv: "dca8e6be116b55b8d75d2fcc3d782180"
server_key_version: "encryptionkey@0"
I20260926 06:24:10.762212 22991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "ef2f80cc690b4703aff5e73fb780295d"
format_stamp: "Formatted at 2026-09-26 06:24:10 on dist-test-slave-pd0t"
server_key: "2cd69a2c3b4ae08e52a4e2bc090f6b7d"
server_key_iv: "dca8e6be116b55b8d75d2fcc3d782180"
server_key_version: "encryptionkey@0"
I20260926 06:24:10.763724 22991 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:24:10.764465 23006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:10.764645 22991 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:24:10.764751 22991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "ef2f80cc690b4703aff5e73fb780295d"
format_stamp: "Formatted at 2026-09-26 06:24:10 on dist-test-slave-pd0t"
server_key: "2cd69a2c3b4ae08e52a4e2bc090f6b7d"
server_key_iv: "dca8e6be116b55b8d75d2fcc3d782180"
server_key_version: "encryptionkey@0"
I20260926 06:24:10.764835 22991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:10.789325 22991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:10.791409 22991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:10.791584 22991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:10.795814 22991 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:40799
I20260926 06:24:10.795845 23058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:40799 every 8 connection(s)
I20260926 06:24:10.796211 22991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:24:10.797055 23059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:10.799271 23059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef2f80cc690b4703aff5e73fb780295d: Bootstrap starting.
I20260926 06:24:10.799793 23059 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef2f80cc690b4703aff5e73fb780295d: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:10.800024 23059 log.cc:824] T 00000000000000000000000000000000 P ef2f80cc690b4703aff5e73fb780295d: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:10.800652 23059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef2f80cc690b4703aff5e73fb780295d: No bootstrap required, opened a new log
I20260926 06:24:10.801761 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 22991
I20260926 06:24:10.801877 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:24:10.802083 4906 external_mini_cluster.cc:1428] Setting key 06fcb0061160caa4788ec89623254157
I20260926 06:24:10.802559 23059 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef2f80cc690b4703aff5e73fb780295d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef2f80cc690b4703aff5e73fb780295d" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 40799 } }
I20260926 06:24:10.802773 23059 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef2f80cc690b4703aff5e73fb780295d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:10.802809 23059 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef2f80cc690b4703aff5e73fb780295d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef2f80cc690b4703aff5e73fb780295d, State: Initialized, Role: FOLLOWER
I20260926 06:24:10.802954 23059 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef2f80cc690b4703aff5e73fb780295d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef2f80cc690b4703aff5e73fb780295d" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 40799 } }
I20260926 06:24:10.803051 23059 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef2f80cc690b4703aff5e73fb780295d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:24:10.803095 23059 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef2f80cc690b4703aff5e73fb780295d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:24:10.803143 23059 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef2f80cc690b4703aff5e73fb780295d [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:10.804263 23059 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef2f80cc690b4703aff5e73fb780295d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef2f80cc690b4703aff5e73fb780295d" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 40799 } }
I20260926 06:24:10.804383 23059 leader_election.cc:302] T 00000000000000000000000000000000 P ef2f80cc690b4703aff5e73fb780295d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef2f80cc690b4703aff5e73fb780295d; no voters:
I20260926 06:24:10.804543 23059 leader_election.cc:288] T 00000000000000000000000000000000 P ef2f80cc690b4703aff5e73fb780295d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:24:10.804594 23064 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef2f80cc690b4703aff5e73fb780295d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:24:10.804729 23064 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef2f80cc690b4703aff5e73fb780295d [term 1 LEADER]: Becoming Leader. State: Replica: ef2f80cc690b4703aff5e73fb780295d, State: Running, Role: LEADER
I20260926 06:24:10.804793 23059 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef2f80cc690b4703aff5e73fb780295d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:24:10.804808 23064 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef2f80cc690b4703aff5e73fb780295d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef2f80cc690b4703aff5e73fb780295d" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 40799 } }
I20260926 06:24:10.805125 23064 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef2f80cc690b4703aff5e73fb780295d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef2f80cc690b4703aff5e73fb780295d. Latest consensus state: current_term: 1 leader_uuid: "ef2f80cc690b4703aff5e73fb780295d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef2f80cc690b4703aff5e73fb780295d" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 40799 } } }
I20260926 06:24:10.805194 23064 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef2f80cc690b4703aff5e73fb780295d [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:10.805136 23066 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef2f80cc690b4703aff5e73fb780295d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef2f80cc690b4703aff5e73fb780295d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef2f80cc690b4703aff5e73fb780295d" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 40799 } } }
I20260926 06:24:10.805383 23066 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef2f80cc690b4703aff5e73fb780295d [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:10.805387 23069 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:24:10.806228 23069 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:24:10.808001 23069 catalog_manager.cc:1347] Generated new cluster ID: 82816c87f3fc41f58abc445941354d64
I20260926 06:24:10.808049 23069 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:24:10 dist-test-slave-pd0t krb5kdc[22978](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:24:10.813436 23063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:10.802626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50452 (local address 127.4.202.190:40799)
0926 06:24:10.802869 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:10.802879 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:10.802916 (+ 37us) server_negotiation.cc:407] Connection header received
0926 06:24:10.803054 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:10.803076 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:10.803131 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:10.803263 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:10.804092 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:10.805407 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:10.806484 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:10.807139 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:10.807239 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:10.809969 (+ 2730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:10.809996 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:10.809997 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:10.810008 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:10.811834 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:10.812267 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:10.812270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:10.812270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:10.812387 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:10.812608 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:10.812609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:10.812609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:10.812883 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:10.813026 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:10.813147 (+ 121us) server_negotiation.cc:299] Negotiation successful
0926 06:24:10.813263 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":96,"threads_started":1}
I20260926 06:24:10.827220 23069 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:24:10.827693 23069 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:24:10.830658 23069 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef2f80cc690b4703aff5e73fb780295d: Generated new TSK 0
I20260926 06:24:10.830832 23069 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.202.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
I20260926 06:24:10.857264 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.129:0
--local_ip_for_outbound_sockets=127.4.202.129
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:40799
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:34467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:24:10.936491 23085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:10.936712 23085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:10.936750 23085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:10.937929 23085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:10.937992 23085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:10.938036 23085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.129
I20260926 06:24:10.939489 23085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:34467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:40799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.129
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:10.939744 23085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:10.939965 23085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:10.944819 23091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:10.944902 23093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:10.944890 23090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:10.945066 23085 server_base.cc:1034] running on GCE node
I20260926 06:24:10.945294 23085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:10.945540 23085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:10.946705 23085 hybrid_clock.cc:648] HybridClock initialized: now 1790403850946691 us; error 43 us; skew 500 ppm
Sep 26 06:24:10 dist-test-slave-pd0t krb5kdc[22978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403850, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:10.949151 23085 init.cc:377] Logged in from keytab as kudu/127.4.202.129@KRBTEST.COM (short username kudu)
I20260926 06:24:10.949517 23085 webserver.cc:460] Webserver started at http://127.4.202.129:40261/ using document root <none> and password file <none>
I20260926 06:24:10.949690 23085 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:10.949728 23085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:10.949811 23085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:10.950820 23085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-0/data/instance:
uuid: "302e5d1c7c3d421c98b339cd9d43f3f4"
format_stamp: "Formatted at 2026-09-26 06:24:10 on dist-test-slave-pd0t"
server_key: "fe7c68f2daaff3b45fc9563dfeb661a3"
server_key_iv: "1acd92ec047f36943ae819dd7d46b731"
server_key_version: "encryptionkey@0"
I20260926 06:24:10.951118 23085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance:
uuid: "302e5d1c7c3d421c98b339cd9d43f3f4"
format_stamp: "Formatted at 2026-09-26 06:24:10 on dist-test-slave-pd0t"
server_key: "fe7c68f2daaff3b45fc9563dfeb661a3"
server_key_iv: "1acd92ec047f36943ae819dd7d46b731"
server_key_version: "encryptionkey@0"
I20260926 06:24:10.952485 23085 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:24:10.953157 23100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:10.953341 23085 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:24:10.953409 23085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-0/wal
uuid: "302e5d1c7c3d421c98b339cd9d43f3f4"
format_stamp: "Formatted at 2026-09-26 06:24:10 on dist-test-slave-pd0t"
server_key: "fe7c68f2daaff3b45fc9563dfeb661a3"
server_key_iv: "1acd92ec047f36943ae819dd7d46b731"
server_key_version: "encryptionkey@0"
I20260926 06:24:10.953460 23085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:10.975462 23085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:10.977278 23085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:10.977387 23085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:10.977556 23085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:24:10.977855 23085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:24:10.977888 23085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:10.977908 23085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:24:10.977955 23085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:10.983861 23085 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.129:41253
I20260926 06:24:10.983945 23213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.129:41253 every 8 connection(s)
I20260926 06:24:10.984234 23085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
Sep 26 06:24:10 dist-test-slave-pd0t krb5kdc[22978](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403850, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:24:10.991851 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 23085
I20260926 06:24:10.991978 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance
I20260926 06:24:10.992193 4906 external_mini_cluster.cc:1428] Setting key d45642d8f085d99e75e37c17d49c4b89
I20260926 06:24:10.993395 23063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:10.984941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:41153 (local address 127.4.202.190:40799)
0926 06:24:10.984987 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:10.984989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:10.985208 (+ 219us) server_negotiation.cc:407] Connection header received
0926 06:24:10.985731 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:10.985732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:10.985744 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:10.985805 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:10.986782 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:10.987296 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:10.988288 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:10.988496 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:10.988580 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:10.990754 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:10.990755 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:10.990755 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:10.990761 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:10.991884 (+ 1123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:10.992346 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:10.992347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:10.992348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:10.992407 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:10.992724 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:10.992725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:10.992725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:10.992864 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:10.992939 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:10.993213 (+ 274us) server_negotiation.cc:299] Negotiation successful
0926 06:24:10.993297 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 06:24:10.993605 23216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:10.984907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:40799 (local address 127.4.202.129:41153)
0926 06:24:10.985089 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:10.985123 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:10.985622 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:10.985871 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:10.985875 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:10.986441 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:10.986681 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:10.986687 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:10.987393 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:10.987394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:10.988183 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:10.988185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:10.988591 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:10.988592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:10.988721 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:10.989229 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:24:10.989236 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:10.990672 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:10.992002 (+ 1330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:10.992008 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:10.992010 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:10.992275 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:10.992520 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:10.992522 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:10.992522 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:10.992657 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:10.992943 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:10.992950 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:10.993083 (+ 133us) client_negotiation.cc:769] Sending connection context
0926 06:24:10.993190 (+ 107us) client_negotiation.cc:240] Negotiation successful
0926 06:24:10.993351 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":94,"threads_started":1}
I20260926 06:24:10.994078 23214 heartbeater.cc:347] Connected to a master server at 127.4.202.190:40799
I20260926 06:24:10.994176 23214 heartbeater.cc:464] Registering TS with master...
I20260926 06:24:10.994359 23214 heartbeater.cc:511] Master 127.4.202.190:40799 requested a full tablet report, sending...
I20260926 06:24:10.994905 23023 ts_manager.cc:194] Registered new tserver with Master: 302e5d1c7c3d421c98b339cd9d43f3f4 (127.4.202.129:41253)
I20260926 06:24:10.995524 23023 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.129@KRBTEST.COM'} at 127.4.202.129:41153
WARNING: no policy specified for kudu/127.4.202.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
I20260926 06:24:11.017299 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.130:0
--local_ip_for_outbound_sockets=127.4.202.130
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:40799
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:34467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:24:11.095561 23219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:11.095741 23219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:11.095767 23219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:11.096836 23219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:11.096868 23219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:11.096915 23219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.130
I20260926 06:24:11.098289 23219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:34467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:40799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.130
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:11.098455 23219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:11.098636 23219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:11.103348 23225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:11.103407 23224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:11.103574 23219 server_base.cc:1034] running on GCE node
W20260926 06:24:11.103348 23227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:11.103829 23219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:11.104097 23219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:11.105286 23219 hybrid_clock.cc:648] HybridClock initialized: now 1790403851105274 us; error 41 us; skew 500 ppm
Sep 26 06:24:11 dist-test-slave-pd0t krb5kdc[22978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403851, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:11.107781 23219 init.cc:377] Logged in from keytab as kudu/127.4.202.130@KRBTEST.COM (short username kudu)
I20260926 06:24:11.108155 23219 webserver.cc:460] Webserver started at http://127.4.202.130:37685/ using document root <none> and password file <none>
I20260926 06:24:11.108332 23219 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:11.108368 23219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:11.108450 23219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:11.109396 23219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-1/data/instance:
uuid: "ff806339cb304a429cf789f8d212b6c4"
format_stamp: "Formatted at 2026-09-26 06:24:11 on dist-test-slave-pd0t"
server_key: "1aa6904c26282eceaf38ec2388c7164d"
server_key_iv: "32ebcc2b9309fd78c51e665eb0cdf06a"
server_key_version: "encryptionkey@0"
I20260926 06:24:11.109664 23219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance:
uuid: "ff806339cb304a429cf789f8d212b6c4"
format_stamp: "Formatted at 2026-09-26 06:24:11 on dist-test-slave-pd0t"
server_key: "1aa6904c26282eceaf38ec2388c7164d"
server_key_iv: "32ebcc2b9309fd78c51e665eb0cdf06a"
server_key_version: "encryptionkey@0"
I20260926 06:24:11.110822 23219 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:24:11.111518 23234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:11.111686 23219 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:24:11.111778 23219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-1/wal
uuid: "ff806339cb304a429cf789f8d212b6c4"
format_stamp: "Formatted at 2026-09-26 06:24:11 on dist-test-slave-pd0t"
server_key: "1aa6904c26282eceaf38ec2388c7164d"
server_key_iv: "32ebcc2b9309fd78c51e665eb0cdf06a"
server_key_version: "encryptionkey@0"
I20260926 06:24:11.111855 23219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:11.133678 23219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:11.135741 23219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:11.135900 23219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:11.136153 23219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:24:11.136461 23219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:24:11.136519 23219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:11.136559 23219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:24:11.136587 23219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:11.142273 23219 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.130:37575
I20260926 06:24:11.142313 23347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.130:37575 every 8 connection(s)
I20260926 06:24:11.142696 23219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
Sep 26 06:24:11 dist-test-slave-pd0t krb5kdc[22978](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403851, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:24:11.151432 23063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:11.143377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:41275 (local address 127.4.202.190:40799)
0926 06:24:11.143462 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:11.143465 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:11.143627 (+ 162us) server_negotiation.cc:407] Connection header received
0926 06:24:11.144305 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:11.144306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:11.144316 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:11.144367 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:11.145202 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.145682 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.146638 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.146849 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.146903 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:11.149004 (+ 2101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:11.149005 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:11.149005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:11.149011 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:11.150241 (+ 1230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:11.150640 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:11.150641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:11.150642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:11.150706 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:11.150915 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:11.150916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:11.150916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:11.151031 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:11.151081 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:11.151303 (+ 222us) server_negotiation.cc:299] Negotiation successful
0926 06:24:11.151369 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 06:24:11.151676 23350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:11.143331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:40799 (local address 127.4.202.130:41275)
0926 06:24:11.143512 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:11.143545 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:11.144207 (+ 662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:11.144418 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:11.144421 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:11.144941 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:11.145126 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:11.145130 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.145756 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.145757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:11.146535 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:11.146537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.146939 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.146940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:11.147072 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:11.147492 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:24:11.147500 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:11.148898 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:11.150314 (+ 1416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:11.150319 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:11.150321 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:11.150540 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:11.150764 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:11.150765 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:11.150765 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:11.150861 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:11.151086 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:11.151090 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:11.151197 (+ 107us) client_negotiation.cc:769] Sending connection context
0926 06:24:11.151277 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 06:24:11.151392 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":89,"threads_started":1}
I20260926 06:24:11.151830 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 23219
I20260926 06:24:11.151981 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance
I20260926 06:24:11.152235 4906 external_mini_cluster.cc:1428] Setting key 308cba660c0204e48512c609a2ed3c67
I20260926 06:24:11.152222 23348 heartbeater.cc:347] Connected to a master server at 127.4.202.190:40799
I20260926 06:24:11.152315 23348 heartbeater.cc:464] Registering TS with master...
I20260926 06:24:11.152498 23348 heartbeater.cc:511] Master 127.4.202.190:40799 requested a full tablet report, sending...
I20260926 06:24:11.152877 23023 ts_manager.cc:194] Registered new tserver with Master: ff806339cb304a429cf789f8d212b6c4 (127.4.202.130:37575)
I20260926 06:24:11.153265 23023 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.130@KRBTEST.COM'} at 127.4.202.130:41275
WARNING: no policy specified for kudu/127.4.202.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
I20260926 06:24:11.177724 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.131:0
--local_ip_for_outbound_sockets=127.4.202.131
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:40799
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:34467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:24:11.255532 23353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:11.255754 23353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:11.255792 23353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:11.256970 23353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:11.257042 23353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:11.257085 23353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.131
I20260926 06:24:11.258536 23353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:34467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:40799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.131
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:11.258826 23353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:11.259050 23353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:11.263684 23361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:11.263679 23359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:11.263677 23358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:11.263898 23353 server_base.cc:1034] running on GCE node
I20260926 06:24:11.264094 23353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:11.264320 23353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:11.265496 23353 hybrid_clock.cc:648] HybridClock initialized: now 1790403851265492 us; error 33 us; skew 500 ppm
Sep 26 06:24:11 dist-test-slave-pd0t krb5kdc[22978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403851, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:11.267846 23353 init.cc:377] Logged in from keytab as kudu/127.4.202.131@KRBTEST.COM (short username kudu)
I20260926 06:24:11.268268 23353 webserver.cc:460] Webserver started at http://127.4.202.131:40253/ using document root <none> and password file <none>
I20260926 06:24:11.268466 23353 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:11.268504 23353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:11.268611 23353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:11.269503 23353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-2/data/instance:
uuid: "5869dc9e2b714e0e81bd603062c1a47e"
format_stamp: "Formatted at 2026-09-26 06:24:11 on dist-test-slave-pd0t"
server_key: "23545b86fb4694ee3257fb365b74998a"
server_key_iv: "f2ce61c2f301ed7d7b42554de7555265"
server_key_version: "encryptionkey@0"
I20260926 06:24:11.269775 23353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance:
uuid: "5869dc9e2b714e0e81bd603062c1a47e"
format_stamp: "Formatted at 2026-09-26 06:24:11 on dist-test-slave-pd0t"
server_key: "23545b86fb4694ee3257fb365b74998a"
server_key_iv: "f2ce61c2f301ed7d7b42554de7555265"
server_key_version: "encryptionkey@0"
I20260926 06:24:11.271168 23353 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:24:11.271853 23368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:11.272051 23353 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:24:11.272119 23353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-2/wal
uuid: "5869dc9e2b714e0e81bd603062c1a47e"
format_stamp: "Formatted at 2026-09-26 06:24:11 on dist-test-slave-pd0t"
server_key: "23545b86fb4694ee3257fb365b74998a"
server_key_iv: "f2ce61c2f301ed7d7b42554de7555265"
server_key_version: "encryptionkey@0"
I20260926 06:24:11.272169 23353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:11.292325 23353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:11.294143 23353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:11.294301 23353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:11.294516 23353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:24:11.294857 23353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:24:11.294909 23353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:11.294947 23353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:24:11.294977 23353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:11.300730 23353 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.131:42113
I20260926 06:24:11.300822 23481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.131:42113 every 8 connection(s)
I20260926 06:24:11.301098 23353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
I20260926 06:24:11.302345 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 23353
I20260926 06:24:11.302467 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance
I20260926 06:24:11.302687 4906 external_mini_cluster.cc:1428] Setting key 097e71acd16cbec4187dd11c715eb3a0
Sep 26 06:24:11 dist-test-slave-pd0t krb5kdc[22978](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403851, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:24:11.310343 23063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:11.301755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:37235 (local address 127.4.202.190:40799)
0926 06:24:11.301822 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:11.301824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:11.301992 (+ 168us) server_negotiation.cc:407] Connection header received
0926 06:24:11.302511 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:11.302513 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:11.302525 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:11.302606 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:11.303734 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.304222 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.305307 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.305587 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.305669 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:11.307803 (+ 2134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:11.307804 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:11.307805 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:11.307811 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:11.309040 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:11.309429 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:11.309431 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:11.309432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:11.309488 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:11.309721 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:11.309722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:11.309722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:11.309831 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:11.309901 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:11.310182 (+ 281us) server_negotiation.cc:299] Negotiation successful
0926 06:24:11.310255 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 06:24:11.310578 23484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:11.301717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:40799 (local address 127.4.202.131:37235)
0926 06:24:11.301863 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:11.301901 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:11.302384 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:11.302667 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:11.302671 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:11.303389 (+ 718us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:11.303630 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:11.303636 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.304310 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.304312 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:11.305224 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:11.305227 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.305674 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.305676 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:11.305816 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:11.306338 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:24:11.306345 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:11.307693 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:11.309127 (+ 1434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:11.309133 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:11.309135 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:11.309340 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:11.309554 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:11.309555 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:11.309555 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:11.309653 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:11.309907 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:11.309914 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:11.310035 (+ 121us) client_negotiation.cc:769] Sending connection context
0926 06:24:11.310157 (+ 122us) client_negotiation.cc:240] Negotiation successful
0926 06:24:11.310303 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":80,"threads_started":1}
I20260926 06:24:11.311036 23482 heartbeater.cc:347] Connected to a master server at 127.4.202.190:40799
I20260926 06:24:11.311125 23482 heartbeater.cc:464] Registering TS with master...
I20260926 06:24:11.311251 23482 heartbeater.cc:511] Master 127.4.202.190:40799 requested a full tablet report, sending...
I20260926 06:24:11.311566 23023 ts_manager.cc:194] Registered new tserver with Master: 5869dc9e2b714e0e81bd603062c1a47e (127.4.202.131:42113)
I20260926 06:24:11.311925 23023 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.131@KRBTEST.COM'} at 127.4.202.131:37235
I20260926 06:24:11.314343 4906 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:24:11.321762 23063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:11.315660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50454 (local address 127.4.202.190:40799)
0926 06:24:11.315697 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:11.315699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:11.315816 (+ 117us) server_negotiation.cc:407] Connection header received
0926 06:24:11.315851 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:11.315851 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:11.315859 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:11.315902 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:11.316384 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.316855 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.317979 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.318218 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.318282 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:11.319473 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:11.319474 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:11.319474 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:11.319480 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:11.320787 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:11.321150 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:11.321151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:11.321151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:11.321200 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:11.321349 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:11.321350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:11.321350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:11.321451 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:11.321523 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:11.321600 (+ 77us) server_negotiation.cc:299] Negotiation successful
0926 06:24:11.321644 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 06:24:11.323413 23023 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50454:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:24:11.324908 23023 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:24:11.331161 23491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:11.327246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:37575 (local address 127.0.0.1:33768)
0926 06:24:11.327418 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:11.327428 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:11.327467 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:11.327884 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:11.327885 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:11.327888 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:24:11.327945 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:11.327947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.328849 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.328850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:11.330046 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:11.330057 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.330830 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.330831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:11.330917 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:11.330929 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 06:24:11.331013 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 06:24:11.331068 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":1,"thread_start_us":62,"threads_started":1}
I20260926 06:24:11.331249 23494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:11.327240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49506 (local address 127.4.202.129:41253)
0926 06:24:11.327546 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:11.327550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:11.327561 (+ 11us) server_negotiation.cc:407] Connection header received
0926 06:24:11.327611 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:11.327613 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:11.327631 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:11.327711 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:24:11.327890 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.329000 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.330126 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.330892 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.330984 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:11.331037 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:11.331124 (+ 87us) server_negotiation.cc:299] Negotiation successful
0926 06:24:11.331166 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":101,"threads_started":1}
I20260926 06:24:11.331380 23493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:11.327072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:41253 (local address 127.0.0.1:49506)
0926 06:24:11.327365 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:11.327375 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:11.327429 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:11.327724 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:11.327724 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:11.327726 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:24:11.327806 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:11.327808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.329131 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.329132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:11.330027 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:11.330030 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.330980 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.330980 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:11.331037 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:11.331040 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:24:11.331110 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 06:24:11.331140 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":21,"thread_start_us":67,"threads_started":1}
I20260926 06:24:11.331485 23495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:11.327439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33768 (local address 127.4.202.130:37575)
0926 06:24:11.327609 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:11.327613 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:11.327622 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:24:11.327658 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:11.327659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:11.327674 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:11.328021 (+ 347us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:24:11.328089 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.328742 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.330165 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.330739 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.330827 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:11.330882 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:11.331362 (+ 480us) server_negotiation.cc:299] Negotiation successful
0926 06:24:11.331405 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":91,"threads_started":1}
I20260926 06:24:11.331602 23492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:11.327090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54596 (local address 127.4.202.131:42113)
0926 06:24:11.327466 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:11.327471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:11.327480 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:24:11.327518 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:11.327520 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:11.327537 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:11.327611 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:24:11.327885 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.329004 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.330247 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.330899 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.330988 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:11.331036 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:11.331474 (+ 438us) server_negotiation.cc:299] Negotiation successful
0926 06:24:11.331523 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":72,"threads_started":1}
I20260926 06:24:11.331321 23490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:11.327042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:42113 (local address 127.0.0.1:54596)
0926 06:24:11.327206 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:11.327242 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:11.327301 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:11.327714 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:11.327717 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:11.327719 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:24:11.327799 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:11.327801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.329112 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.329113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:11.330175 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:11.330177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.331074 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.331075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:11.331141 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:11.331145 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:24:11.331196 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 06:24:11.331239 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":4,"thread_start_us":74,"threads_started":1}
I20260926 06:24:11.332129 23148 tablet_service.cc:1467] Processing CreateTablet for tablet 1214d40280224360ade235f1f21a06b2 (DEFAULT_TABLE table=test-table [id=5169557a567e432ebd35990e9d4c5194]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:24:11.332108 23282 tablet_service.cc:1467] Processing CreateTablet for tablet 1214d40280224360ade235f1f21a06b2 (DEFAULT_TABLE table=test-table [id=5169557a567e432ebd35990e9d4c5194]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:24:11.332285 23416 tablet_service.cc:1467] Processing CreateTablet for tablet 1214d40280224360ade235f1f21a06b2 (DEFAULT_TABLE table=test-table [id=5169557a567e432ebd35990e9d4c5194]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:24:11.332379 23148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1214d40280224360ade235f1f21a06b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:11.332417 23282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1214d40280224360ade235f1f21a06b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:11.332536 23416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1214d40280224360ade235f1f21a06b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:11.334709 23496 tablet_bootstrap.cc:492] T 1214d40280224360ade235f1f21a06b2 P 302e5d1c7c3d421c98b339cd9d43f3f4: Bootstrap starting.
I20260926 06:24:11.335391 23496 tablet_bootstrap.cc:654] T 1214d40280224360ade235f1f21a06b2 P 302e5d1c7c3d421c98b339cd9d43f3f4: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:11.335583 23498 tablet_bootstrap.cc:492] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e: Bootstrap starting.
I20260926 06:24:11.335744 23497 tablet_bootstrap.cc:492] T 1214d40280224360ade235f1f21a06b2 P ff806339cb304a429cf789f8d212b6c4: Bootstrap starting.
I20260926 06:24:11.336021 23496 log.cc:824] T 1214d40280224360ade235f1f21a06b2 P 302e5d1c7c3d421c98b339cd9d43f3f4: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:11.336220 23498 tablet_bootstrap.cc:654] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:11.336460 23497 tablet_bootstrap.cc:654] T 1214d40280224360ade235f1f21a06b2 P ff806339cb304a429cf789f8d212b6c4: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:11.336850 23498 log.cc:824] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:11.336963 23496 tablet_bootstrap.cc:492] T 1214d40280224360ade235f1f21a06b2 P 302e5d1c7c3d421c98b339cd9d43f3f4: No bootstrap required, opened a new log
I20260926 06:24:11.336999 23497 log.cc:824] T 1214d40280224360ade235f1f21a06b2 P ff806339cb304a429cf789f8d212b6c4: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:11.337036 23496 ts_tablet_manager.cc:1397] T 1214d40280224360ade235f1f21a06b2 P 302e5d1c7c3d421c98b339cd9d43f3f4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:24:11.337651 23498 tablet_bootstrap.cc:492] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e: No bootstrap required, opened a new log
I20260926 06:24:11.337718 23498 ts_tablet_manager.cc:1397] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:24:11.339176 23497 tablet_bootstrap.cc:492] T 1214d40280224360ade235f1f21a06b2 P ff806339cb304a429cf789f8d212b6c4: No bootstrap required, opened a new log
I20260926 06:24:11.339087 23496 raft_consensus.cc:348] T 1214d40280224360ade235f1f21a06b2 P 302e5d1c7c3d421c98b339cd9d43f3f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5869dc9e2b714e0e81bd603062c1a47e" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 42113 } } peers { permanent_uuid: "ff806339cb304a429cf789f8d212b6c4" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 37575 } } peers { permanent_uuid: "302e5d1c7c3d421c98b339cd9d43f3f4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 41253 } }
I20260926 06:24:11.339246 23496 raft_consensus.cc:374] T 1214d40280224360ade235f1f21a06b2 P 302e5d1c7c3d421c98b339cd9d43f3f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:11.339260 23497 ts_tablet_manager.cc:1397] T 1214d40280224360ade235f1f21a06b2 P ff806339cb304a429cf789f8d212b6c4: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260926 06:24:11.339293 23496 raft_consensus.cc:729] T 1214d40280224360ade235f1f21a06b2 P 302e5d1c7c3d421c98b339cd9d43f3f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 302e5d1c7c3d421c98b339cd9d43f3f4, State: Initialized, Role: FOLLOWER
I20260926 06:24:11.339437 23496 consensus_queue.cc:260] T 1214d40280224360ade235f1f21a06b2 P 302e5d1c7c3d421c98b339cd9d43f3f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5869dc9e2b714e0e81bd603062c1a47e" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 42113 } } peers { permanent_uuid: "ff806339cb304a429cf789f8d212b6c4" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 37575 } } peers { permanent_uuid: "302e5d1c7c3d421c98b339cd9d43f3f4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 41253 } }
I20260926 06:24:11.339707 23496 ts_tablet_manager.cc:1428] T 1214d40280224360ade235f1f21a06b2 P 302e5d1c7c3d421c98b339cd9d43f3f4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 06:24:11.339759 23214 heartbeater.cc:503] Master 127.4.202.190:40799 was elected leader, sending a full tablet report...
I20260926 06:24:11.339676 23498 raft_consensus.cc:348] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5869dc9e2b714e0e81bd603062c1a47e" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 42113 } } peers { permanent_uuid: "ff806339cb304a429cf789f8d212b6c4" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 37575 } } peers { permanent_uuid: "302e5d1c7c3d421c98b339cd9d43f3f4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 41253 } }
I20260926 06:24:11.339830 23498 raft_consensus.cc:374] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:11.339869 23498 raft_consensus.cc:729] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5869dc9e2b714e0e81bd603062c1a47e, State: Initialized, Role: FOLLOWER
I20260926 06:24:11.339962 23498 consensus_queue.cc:260] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5869dc9e2b714e0e81bd603062c1a47e" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 42113 } } peers { permanent_uuid: "ff806339cb304a429cf789f8d212b6c4" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 37575 } } peers { permanent_uuid: "302e5d1c7c3d421c98b339cd9d43f3f4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 41253 } }
I20260926 06:24:11.340207 23498 ts_tablet_manager.cc:1428] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 06:24:11.340234 23482 heartbeater.cc:503] Master 127.4.202.190:40799 was elected leader, sending a full tablet report...
I20260926 06:24:11.341266 23497 raft_consensus.cc:348] T 1214d40280224360ade235f1f21a06b2 P ff806339cb304a429cf789f8d212b6c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5869dc9e2b714e0e81bd603062c1a47e" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 42113 } } peers { permanent_uuid: "ff806339cb304a429cf789f8d212b6c4" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 37575 } } peers { permanent_uuid: "302e5d1c7c3d421c98b339cd9d43f3f4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 41253 } }
I20260926 06:24:11.341416 23497 raft_consensus.cc:374] T 1214d40280224360ade235f1f21a06b2 P ff806339cb304a429cf789f8d212b6c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:11.341465 23497 raft_consensus.cc:729] T 1214d40280224360ade235f1f21a06b2 P ff806339cb304a429cf789f8d212b6c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff806339cb304a429cf789f8d212b6c4, State: Initialized, Role: FOLLOWER
I20260926 06:24:11.341564 23497 consensus_queue.cc:260] T 1214d40280224360ade235f1f21a06b2 P ff806339cb304a429cf789f8d212b6c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5869dc9e2b714e0e81bd603062c1a47e" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 42113 } } peers { permanent_uuid: "ff806339cb304a429cf789f8d212b6c4" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 37575 } } peers { permanent_uuid: "302e5d1c7c3d421c98b339cd9d43f3f4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 41253 } }
I20260926 06:24:11.341832 23497 ts_tablet_manager.cc:1428] T 1214d40280224360ade235f1f21a06b2 P ff806339cb304a429cf789f8d212b6c4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 06:24:11.341846 23348 heartbeater.cc:503] Master 127.4.202.190:40799 was elected leader, sending a full tablet report...
W20260926 06:24:11.393239 23349 tablet.cc:2369] T 1214d40280224360ade235f1f21a06b2 P ff806339cb304a429cf789f8d212b6c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 06:24:11.485021 23215 tablet.cc:2369] T 1214d40280224360ade235f1f21a06b2 P 302e5d1c7c3d421c98b339cd9d43f3f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 06:24:11.551620 23483 tablet.cc:2369] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:24:11.637013 23503 raft_consensus.cc:482] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:24:11.637177 23503 raft_consensus.cc:504] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5869dc9e2b714e0e81bd603062c1a47e" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 42113 } } peers { permanent_uuid: "ff806339cb304a429cf789f8d212b6c4" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 37575 } } peers { permanent_uuid: "302e5d1c7c3d421c98b339cd9d43f3f4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 41253 } }
I20260926 06:24:11.637594 23503 leader_election.cc:288] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff806339cb304a429cf789f8d212b6c4 (127.4.202.130:37575), 302e5d1c7c3d421c98b339cd9d43f3f4 (127.4.202.129:41253)
I20260926 06:24:11.641332 23494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:11.637951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:38319 (local address 127.4.202.129:41253)
0926 06:24:11.638009 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:11.638011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:11.638188 (+ 177us) server_negotiation.cc:407] Connection header received
0926 06:24:11.638217 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:11.638218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:11.638228 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:11.638284 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:24:11.638436 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.639161 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.640415 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.640967 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.641047 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:11.641056 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:11.641226 (+ 170us) server_negotiation.cc:299] Negotiation successful
0926 06:24:11.641262 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 06:24:11.641332 23495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:11.637841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:39281 (local address 127.4.202.130:37575)
0926 06:24:11.637888 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:11.637890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:11.637922 (+ 32us) server_negotiation.cc:407] Connection header received
0926 06:24:11.637952 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:11.637953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:11.637962 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:11.638020 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:24:11.638210 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.639046 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.640416 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.640988 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.641050 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:11.641057 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:11.641226 (+ 169us) server_negotiation.cc:299] Negotiation successful
0926 06:24:11.641264 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 06:24:11.641340 23484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:11.637776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:37575 (local address 127.4.202.131:39281)
0926 06:24:11.637824 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:11.637837 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:11.637884 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:11.638037 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:11.638038 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:11.638040 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:24:11.638124 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:11.638126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.639122 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.639123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:11.640310 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:11.640313 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.641065 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.641065 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:11.641145 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:11.641147 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:24:11.641202 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 06:24:11.641245 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":7}
I20260926 06:24:11.641346 23505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:11.637863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:41253 (local address 127.4.202.131:38319)
0926 06:24:11.638092 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:11.638104 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:11.638160 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:11.638298 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:11.638299 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:11.638300 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:24:11.638353 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:11.638354 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.639229 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.639230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:11.640311 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:11.640313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.641065 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.641065 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:11.641142 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:11.641146 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:24:11.641202 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 06:24:11.641245 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":4,"thread_start_us":148,"threads_started":1}
I20260926 06:24:11.641705 23302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1214d40280224360ade235f1f21a06b2" candidate_uuid: "5869dc9e2b714e0e81bd603062c1a47e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff806339cb304a429cf789f8d212b6c4" is_pre_election: true
I20260926 06:24:11.641710 23168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1214d40280224360ade235f1f21a06b2" candidate_uuid: "5869dc9e2b714e0e81bd603062c1a47e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "302e5d1c7c3d421c98b339cd9d43f3f4" is_pre_election: true
I20260926 06:24:11.641872 23302 raft_consensus.cc:2454] T 1214d40280224360ade235f1f21a06b2 P ff806339cb304a429cf789f8d212b6c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5869dc9e2b714e0e81bd603062c1a47e in term 0.
I20260926 06:24:11.641887 23168 raft_consensus.cc:2454] T 1214d40280224360ade235f1f21a06b2 P 302e5d1c7c3d421c98b339cd9d43f3f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5869dc9e2b714e0e81bd603062c1a47e in term 0.
I20260926 06:24:11.642138 23372 leader_election.cc:302] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 302e5d1c7c3d421c98b339cd9d43f3f4, 5869dc9e2b714e0e81bd603062c1a47e, ff806339cb304a429cf789f8d212b6c4; no voters:
I20260926 06:24:11.642258 23503 raft_consensus.cc:2781] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:24:11.642310 23503 raft_consensus.cc:482] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:24:11.642339 23503 raft_consensus.cc:3037] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:11.643011 23503 raft_consensus.cc:504] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5869dc9e2b714e0e81bd603062c1a47e" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 42113 } } peers { permanent_uuid: "ff806339cb304a429cf789f8d212b6c4" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 37575 } } peers { permanent_uuid: "302e5d1c7c3d421c98b339cd9d43f3f4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 41253 } }
I20260926 06:24:11.643137 23503 leader_election.cc:288] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e [CANDIDATE]: Term 1 election: Requested vote from peers ff806339cb304a429cf789f8d212b6c4 (127.4.202.130:37575), 302e5d1c7c3d421c98b339cd9d43f3f4 (127.4.202.129:41253)
I20260926 06:24:11.643246 23302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1214d40280224360ade235f1f21a06b2" candidate_uuid: "5869dc9e2b714e0e81bd603062c1a47e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff806339cb304a429cf789f8d212b6c4"
I20260926 06:24:11.643307 23168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1214d40280224360ade235f1f21a06b2" candidate_uuid: "5869dc9e2b714e0e81bd603062c1a47e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "302e5d1c7c3d421c98b339cd9d43f3f4"
I20260926 06:24:11.643347 23302 raft_consensus.cc:3037] T 1214d40280224360ade235f1f21a06b2 P ff806339cb304a429cf789f8d212b6c4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:11.643380 23168 raft_consensus.cc:3037] T 1214d40280224360ade235f1f21a06b2 P 302e5d1c7c3d421c98b339cd9d43f3f4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:11.644057 23302 raft_consensus.cc:2454] T 1214d40280224360ade235f1f21a06b2 P ff806339cb304a429cf789f8d212b6c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5869dc9e2b714e0e81bd603062c1a47e in term 1.
I20260926 06:24:11.644109 23168 raft_consensus.cc:2454] T 1214d40280224360ade235f1f21a06b2 P 302e5d1c7c3d421c98b339cd9d43f3f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5869dc9e2b714e0e81bd603062c1a47e in term 1.
I20260926 06:24:11.644274 23369 leader_election.cc:302] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5869dc9e2b714e0e81bd603062c1a47e, ff806339cb304a429cf789f8d212b6c4; no voters:
I20260926 06:24:11.644387 23503 raft_consensus.cc:2781] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:24:11.644536 23503 raft_consensus.cc:686] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e [term 1 LEADER]: Becoming Leader. State: Replica: 5869dc9e2b714e0e81bd603062c1a47e, State: Running, Role: LEADER
I20260926 06:24:11.644629 23503 consensus_queue.cc:237] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5869dc9e2b714e0e81bd603062c1a47e" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 42113 } } peers { permanent_uuid: "ff806339cb304a429cf789f8d212b6c4" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 37575 } } peers { permanent_uuid: "302e5d1c7c3d421c98b339cd9d43f3f4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 41253 } }
I20260926 06:24:11.645340 23022 catalog_manager.cc:5495] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e reported cstate change: term changed from 0 to 1, leader changed from <none> to 5869dc9e2b714e0e81bd603062c1a47e (127.4.202.131). New cstate: current_term: 1 leader_uuid: "5869dc9e2b714e0e81bd603062c1a47e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5869dc9e2b714e0e81bd603062c1a47e" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff806339cb304a429cf789f8d212b6c4" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 37575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "302e5d1c7c3d421c98b339cd9d43f3f4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 41253 } health_report { overall_health: UNKNOWN } } }
I20260926 06:24:11.686296 23492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:11.683966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54610 (local address 127.4.202.131:42113)
0926 06:24:11.684021 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:11.684023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:11.684031 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:24:11.684080 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:11.684081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:11.684097 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:11.684155 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:24:11.684268 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.684740 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.685447 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.685633 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.685700 (+ 67us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:11.685848 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:24:11.685930 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:24:11.686028 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:11.686143 (+ 115us) server_negotiation.cc:299] Negotiation successful
0926 06:24:11.686176 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 06:24:11.688100 23302 raft_consensus.cc:1261] T 1214d40280224360ade235f1f21a06b2 P ff806339cb304a429cf789f8d212b6c4 [term 1 FOLLOWER]: Refusing update from remote peer 5869dc9e2b714e0e81bd603062c1a47e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:24:11.688099 23168 raft_consensus.cc:1261] T 1214d40280224360ade235f1f21a06b2 P 302e5d1c7c3d421c98b339cd9d43f3f4 [term 1 FOLLOWER]: Refusing update from remote peer 5869dc9e2b714e0e81bd603062c1a47e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:24:11.688485 23506 consensus_queue.cc:1035] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff806339cb304a429cf789f8d212b6c4" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 37575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:24:11.688620 23503 consensus_queue.cc:1035] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e [LEADER]: Connected to new peer: Peer: permanent_uuid: "302e5d1c7c3d421c98b339cd9d43f3f4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 41253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:24:11.691066 23509 mvcc.cc:205] Tried to move back new op lower bound from 7333494176511881216 to 7333494176336904192. Current Snapshot: MvccSnapshot[applied={T|T < 7333494176511881216}]
I20260926 06:24:11.691381 23510 mvcc.cc:205] Tried to move back new op lower bound from 7333494176511881216 to 7333494176336904192. Current Snapshot: MvccSnapshot[applied={T|T < 7333494176511881216}]
I20260926 06:24:11.691500 23511 mvcc.cc:205] Tried to move back new op lower bound from 7333494176511881216 to 7333494176336904192. Current Snapshot: MvccSnapshot[applied={T|T < 7333494176511881216}]
I20260926 06:24:11.693740 23021 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:24:11.693827 23021 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:24:11.694664 23021 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ef2f80cc690b4703aff5e73fb780295d: Sending DeleteTablet for 3 replicas of tablet 1214d40280224360ade235f1f21a06b2
I20260926 06:24:11.695068 23148 tablet_service.cc:1514] Processing DeleteTablet for tablet 1214d40280224360ade235f1f21a06b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:24:11 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:49506
I20260926 06:24:11.695086 23416 tablet_service.cc:1514] Processing DeleteTablet for tablet 1214d40280224360ade235f1f21a06b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:24:11 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:54596
I20260926 06:24:11.695086 23282 tablet_service.cc:1514] Processing DeleteTablet for tablet 1214d40280224360ade235f1f21a06b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:24:11 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:33768
I20260926 06:24:11.695294 23519 tablet_replica.cc:330] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e: stopping tablet replica
I20260926 06:24:11.695379 23518 tablet_replica.cc:330] T 1214d40280224360ade235f1f21a06b2 P 302e5d1c7c3d421c98b339cd9d43f3f4: stopping tablet replica
I20260926 06:24:11.695415 23519 raft_consensus.cc:2229] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:24:11.695456 23518 raft_consensus.cc:2229] T 1214d40280224360ade235f1f21a06b2 P 302e5d1c7c3d421c98b339cd9d43f3f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:24:11.695528 23519 raft_consensus.cc:2258] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:24:11.695547 23518 raft_consensus.cc:2258] T 1214d40280224360ade235f1f21a06b2 P 302e5d1c7c3d421c98b339cd9d43f3f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:24:11.695679 23520 tablet_replica.cc:330] T 1214d40280224360ade235f1f21a06b2 P ff806339cb304a429cf789f8d212b6c4: stopping tablet replica
I20260926 06:24:11.695683 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 23085
I20260926 06:24:11.695756 23520 raft_consensus.cc:2229] T 1214d40280224360ade235f1f21a06b2 P ff806339cb304a429cf789f8d212b6c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:24:11.696316 23520 raft_consensus.cc:2258] T 1214d40280224360ade235f1f21a06b2 P ff806339cb304a429cf789f8d212b6c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:24:11.697304 23520 ts_tablet_manager.cc:1905] T 1214d40280224360ade235f1f21a06b2 P ff806339cb304a429cf789f8d212b6c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:24:11.698382 23519 ts_tablet_manager.cc:1905] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:24:11.699674 23520 ts_tablet_manager.cc:1918] T 1214d40280224360ade235f1f21a06b2 P ff806339cb304a429cf789f8d212b6c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:24:11.699749 23520 log.cc:1196] T 1214d40280224360ade235f1f21a06b2 P ff806339cb304a429cf789f8d212b6c4: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-1/wal/wals/1214d40280224360ade235f1f21a06b2
I20260926 06:24:11.700029 23520 ts_tablet_manager.cc:1939] T 1214d40280224360ade235f1f21a06b2 P ff806339cb304a429cf789f8d212b6c4: Deleting consensus metadata
I20260926 06:24:11.700029 23519 ts_tablet_manager.cc:1918] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:24:11.700084 23519 log.cc:1196] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790403591640726-4906-0/minicluster-data/ts-2/wal/wals/1214d40280224360ade235f1f21a06b2
I20260926 06:24:11.700335 23519 ts_tablet_manager.cc:1939] T 1214d40280224360ade235f1f21a06b2 P 5869dc9e2b714e0e81bd603062c1a47e: Deleting consensus metadata
I20260926 06:24:11.700539 23007 catalog_manager.cc:4841] TS ff806339cb304a429cf789f8d212b6c4 (127.4.202.130:37575): tablet 1214d40280224360ade235f1f21a06b2 (table test-table [id=5169557a567e432ebd35990e9d4c5194]) successfully deleted
I20260926 06:24:11.700680 23007 catalog_manager.cc:4841] TS 5869dc9e2b714e0e81bd603062c1a47e (127.4.202.131:42113): tablet 1214d40280224360ade235f1f21a06b2 (table test-table [id=5169557a567e432ebd35990e9d4c5194]) successfully deleted
W20260926 06:24:11.702211 23010 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.202.129:41253 (error 108)
I20260926 06:24:11.702685 23490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:11.702537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:41253 (local address 127.0.0.1:49520)
0926 06:24:11.702616 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:11.702651 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.129:41253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 06:24:11.702724 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 23219
W20260926 06:24:11.702806 23010 catalog_manager.cc:4584] TS 302e5d1c7c3d421c98b339cd9d43f3f4 (127.4.202.129:41253): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1214d40280224360ade235f1f21a06b2: Network error: Client connection negotiation failed: client connection to 127.4.202.129:41253: connect: Connection refused (error 111)
I20260926 06:24:11.708349 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 23353
I20260926 06:24:11.713049 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 22991
2026-09-26T06:24:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2196 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-26T06:24:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:24:11Z Disabled control of system clock
I20260926 06:24:11.739681 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:37959
--webserver_interface=127.4.202.190
--webserver_port=0
--builtin_ntp_servers=127.4.202.148:39953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:37959
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 06:24:11.818127 23525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:11.818322 23525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:11.818342 23525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:11.819527 23525 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:24:11.819566 23525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:11.819579 23525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:11.819592 23525 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:24:11.819604 23525 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:24:11.820813 23525 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:39953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:37959
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:37959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:11.820988 23525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:11.821192 23525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:11.825887 23531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:11.825886 23533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:11.825968 23525 server_base.cc:1034] running on GCE node
W20260926 06:24:11.825914 23530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:11.826411 23525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:11.826757 23525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:11.827909 23525 hybrid_clock.cc:648] HybridClock initialized: now 1790403851827889 us; error 32 us; skew 500 ppm
I20260926 06:24:11.835345 23525 webserver.cc:460] Webserver started at http://127.4.202.190:45155/ using document root <none> and password file <none>
I20260926 06:24:11.835610 23525 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:11.835697 23525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:11.835857 23525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:11.836856 23525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "aa11d9198a004307aff5c8252f206233"
format_stamp: "Formatted at 2026-09-26 06:24:11 on dist-test-slave-pd0t"
server_key: "7d59c7dbe46ed639a6410b66804e58fc"
server_key_iv: "e801d6604bd0bf516571f711e9aadc13"
server_key_version: "encryptionkey@0"
I20260926 06:24:11.837205 23525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "aa11d9198a004307aff5c8252f206233"
format_stamp: "Formatted at 2026-09-26 06:24:11 on dist-test-slave-pd0t"
server_key: "7d59c7dbe46ed639a6410b66804e58fc"
server_key_iv: "e801d6604bd0bf516571f711e9aadc13"
server_key_version: "encryptionkey@0"
I20260926 06:24:11.838421 23525 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:24:11.839192 23539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:11.839367 23525 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:11.839455 23525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "aa11d9198a004307aff5c8252f206233"
format_stamp: "Formatted at 2026-09-26 06:24:11 on dist-test-slave-pd0t"
server_key: "7d59c7dbe46ed639a6410b66804e58fc"
server_key_iv: "e801d6604bd0bf516571f711e9aadc13"
server_key_version: "encryptionkey@0"
I20260926 06:24:11.839542 23525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:11.848230 23525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:11.848487 23525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:11.848626 23525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:11.852403 23525 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:37959
I20260926 06:24:11.852442 23591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:37959 every 8 connection(s)
I20260926 06:24:11.852732 23525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:24:11.853380 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 23525
I20260926 06:24:11.853497 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:24:11.853672 4906 external_mini_cluster.cc:1428] Setting key 5773edf1ce44fc138c6b214caa6472d6
I20260926 06:24:11.853669 23592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:11.856858 23592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa11d9198a004307aff5c8252f206233: Bootstrap starting.
I20260926 06:24:11.857421 23592 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa11d9198a004307aff5c8252f206233: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:11.857699 23592 log.cc:824] T 00000000000000000000000000000000 P aa11d9198a004307aff5c8252f206233: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:11.858485 23592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa11d9198a004307aff5c8252f206233: No bootstrap required, opened a new log
I20260926 06:24:11.858927 23595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:11.854173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53352 (local address 127.4.202.190:37959)
0926 06:24:11.854363 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:11.854369 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:11.854392 (+ 23us) server_negotiation.cc:407] Connection header received
0926 06:24:11.854813 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:11.854819 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:11.854876 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:11.854987 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:11.855209 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.856034 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.857071 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:11.857430 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:11.857560 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:11.858206 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:11.858210 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:11.858211 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:11.858426 (+ 215us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:11.858465 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:11.858491 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:11.858560 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:11.858663 (+ 103us) server_negotiation.cc:299] Negotiation successful
0926 06:24:11.858735 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":82,"threads_started":1}
I20260926 06:24:11.860397 23592 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa11d9198a004307aff5c8252f206233 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa11d9198a004307aff5c8252f206233" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 37959 } }
I20260926 06:24:11.860517 23592 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa11d9198a004307aff5c8252f206233 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:11.860540 23592 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa11d9198a004307aff5c8252f206233 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa11d9198a004307aff5c8252f206233, State: Initialized, Role: FOLLOWER
I20260926 06:24:11.860672 23592 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa11d9198a004307aff5c8252f206233 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa11d9198a004307aff5c8252f206233" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 37959 } }
I20260926 06:24:11.860737 23592 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa11d9198a004307aff5c8252f206233 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:24:11.860757 23592 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa11d9198a004307aff5c8252f206233 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:24:11.860783 23592 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa11d9198a004307aff5c8252f206233 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:11.861367 23592 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa11d9198a004307aff5c8252f206233 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa11d9198a004307aff5c8252f206233" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 37959 } }
I20260926 06:24:11.861472 23592 leader_election.cc:302] T 00000000000000000000000000000000 P aa11d9198a004307aff5c8252f206233 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa11d9198a004307aff5c8252f206233; no voters:
I20260926 06:24:11.861640 23592 leader_election.cc:288] T 00000000000000000000000000000000 P aa11d9198a004307aff5c8252f206233 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:24:11.861696 23597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa11d9198a004307aff5c8252f206233 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:24:11.861868 23597 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa11d9198a004307aff5c8252f206233 [term 1 LEADER]: Becoming Leader. State: Replica: aa11d9198a004307aff5c8252f206233, State: Running, Role: LEADER
I20260926 06:24:11.861927 23592 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa11d9198a004307aff5c8252f206233 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:24:11.861954 23597 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa11d9198a004307aff5c8252f206233 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa11d9198a004307aff5c8252f206233" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 37959 } }
I20260926 06:24:11.862299 23598 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa11d9198a004307aff5c8252f206233 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa11d9198a004307aff5c8252f206233" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa11d9198a004307aff5c8252f206233" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 37959 } } }
I20260926 06:24:11.862299 23599 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa11d9198a004307aff5c8252f206233 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa11d9198a004307aff5c8252f206233. Latest consensus state: current_term: 1 leader_uuid: "aa11d9198a004307aff5c8252f206233" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa11d9198a004307aff5c8252f206233" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 37959 } } }
I20260926 06:24:11.862442 23599 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa11d9198a004307aff5c8252f206233 [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:11.862427 23598 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa11d9198a004307aff5c8252f206233 [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:11.862710 23606 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:24:11.863379 23606 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:24:11.864728 23606 catalog_manager.cc:1347] Generated new cluster ID: 628bcfb909494e43971dd6ff6d92f5c4
I20260926 06:24:11.864789 23606 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:24:11.868916 23606 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:24:11.869349 23606 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:24:11.874244 23606 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa11d9198a004307aff5c8252f206233: Generated new TSK 0
I20260926 06:24:11.874415 23606 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:24:11.881713 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.129:0
--local_ip_for_outbound_sockets=127.4.202.129
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:37959
--builtin_ntp_servers=127.4.202.148:39953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 06:24:11.973568 23616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:11.973785 23616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:11.973819 23616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:11.974983 23616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:11.975035 23616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:11.975075 23616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.129
I20260926 06:24:11.976666 23616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:39953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:37959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.129
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:11.976950 23616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:11.977234 23616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:11.982017 23621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:11.982019 23624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:11.982275 23622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:11.982672 23616 server_base.cc:1034] running on GCE node
I20260926 06:24:11.982867 23616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:11.983119 23616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:11.984298 23616 hybrid_clock.cc:648] HybridClock initialized: now 1790403851984285 us; error 26 us; skew 500 ppm
I20260926 06:24:11.985669 23616 webserver.cc:460] Webserver started at http://127.4.202.129:35183/ using document root <none> and password file <none>
I20260926 06:24:11.985925 23616 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:11.985991 23616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:11.986120 23616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:11.987496 23616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-0/data/instance:
uuid: "81ef1594c8c84c609d92a82300dd90f4"
format_stamp: "Formatted at 2026-09-26 06:24:11 on dist-test-slave-pd0t"
server_key: "b9c77608db33cc991640d129c639c94c"
server_key_iv: "f31cd40c5b9899564f8f23ca2e1bf2ae"
server_key_version: "encryptionkey@0"
I20260926 06:24:11.987886 23616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance:
uuid: "81ef1594c8c84c609d92a82300dd90f4"
format_stamp: "Formatted at 2026-09-26 06:24:11 on dist-test-slave-pd0t"
server_key: "b9c77608db33cc991640d129c639c94c"
server_key_iv: "f31cd40c5b9899564f8f23ca2e1bf2ae"
server_key_version: "encryptionkey@0"
I20260926 06:24:11.989471 23616 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:24:11.990283 23630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:11.990477 23616 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:24:11.990566 23616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-0/wal
uuid: "81ef1594c8c84c609d92a82300dd90f4"
format_stamp: "Formatted at 2026-09-26 06:24:11 on dist-test-slave-pd0t"
server_key: "b9c77608db33cc991640d129c639c94c"
server_key_iv: "f31cd40c5b9899564f8f23ca2e1bf2ae"
server_key_version: "encryptionkey@0"
I20260926 06:24:11.990662 23616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:12.009845 23616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:12.010139 23616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:12.010277 23616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:12.010506 23616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:24:12.010856 23616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:24:12.010900 23616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:12.010928 23616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:24:12.010946 23616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:12.017198 23616 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.129:33561
I20260926 06:24:12.017251 23743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.129:33561 every 8 connection(s)
I20260926 06:24:12.017535 23616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
I20260926 06:24:12.022341 23595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:12.018277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:34597 (local address 127.4.202.190:37959)
0926 06:24:12.018327 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:12.018329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:12.018482 (+ 153us) server_negotiation.cc:407] Connection header received
0926 06:24:12.018924 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:12.018925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:12.018934 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:12.018990 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:12.019723 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.020205 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.021103 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.021283 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.021338 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:12.022045 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:12.022046 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:12.022047 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:12.022051 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:12.022064 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:12.022068 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:12.022112 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:12.022228 (+ 116us) server_negotiation.cc:299] Negotiation successful
0926 06:24:12.022262 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 06:24:12.022549 23746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:12.018173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37959 (local address 127.4.202.129:34597)
0926 06:24:12.018348 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:12.018385 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:12.018843 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:12.019037 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:12.019040 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:12.019048 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:12.019647 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:12.019652 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.020281 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.020282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:12.021024 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:12.021026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.021358 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.021359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:12.021474 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:12.021917 (+ 443us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:24:12.021917 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:12.021947 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:24:12.021966 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:12.022124 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:12.022128 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:12.022131 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:24:12.022209 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 06:24:12.022300 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":91,"threads_started":1}
I20260926 06:24:12.023062 23744 heartbeater.cc:347] Connected to a master server at 127.4.202.190:37959
I20260926 06:24:12.023166 23744 heartbeater.cc:464] Registering TS with master...
I20260926 06:24:12.023335 23744 heartbeater.cc:511] Master 127.4.202.190:37959 requested a full tablet report, sending...
I20260926 06:24:12.023820 23556 ts_manager.cc:194] Registered new tserver with Master: 81ef1594c8c84c609d92a82300dd90f4 (127.4.202.129:33561)
I20260926 06:24:12.024410 23556 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.202.129:34597
I20260926 06:24:12.026754 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 23616
I20260926 06:24:12.026863 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance
I20260926 06:24:12.027051 4906 external_mini_cluster.cc:1428] Setting key 93ed5c22f119e6b33c6afb03ec13e366
I20260926 06:24:12.028079 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.130:0
--local_ip_for_outbound_sockets=127.4.202.130
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:37959
--builtin_ntp_servers=127.4.202.148:39953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 06:24:12.106742 23747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:12.106936 23747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:12.106958 23747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:12.108117 23747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:12.108158 23747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:12.108182 23747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.130
I20260926 06:24:12.109522 23747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:39953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:37959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.130
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:12.109750 23747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:12.110054 23747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:12.114805 23755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:12.114830 23753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:12.114828 23752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:12.115206 23747 server_base.cc:1034] running on GCE node
I20260926 06:24:12.115461 23747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:12.115695 23747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:12.116864 23747 hybrid_clock.cc:648] HybridClock initialized: now 1790403852116846 us; error 30 us; skew 500 ppm
I20260926 06:24:12.118129 23747 webserver.cc:460] Webserver started at http://127.4.202.130:37623/ using document root <none> and password file <none>
I20260926 06:24:12.118430 23747 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:12.118510 23747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:12.118661 23747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:12.119709 23747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-1/data/instance:
uuid: "ac72977b6c814241a3a55fb9b7e5c1e1"
format_stamp: "Formatted at 2026-09-26 06:24:12 on dist-test-slave-pd0t"
server_key: "bb7766d6a9a4129f5ea6da217509e3c9"
server_key_iv: "415caefb718e913226518f4bb446d4b9"
server_key_version: "encryptionkey@0"
I20260926 06:24:12.120132 23747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance:
uuid: "ac72977b6c814241a3a55fb9b7e5c1e1"
format_stamp: "Formatted at 2026-09-26 06:24:12 on dist-test-slave-pd0t"
server_key: "bb7766d6a9a4129f5ea6da217509e3c9"
server_key_iv: "415caefb718e913226518f4bb446d4b9"
server_key_version: "encryptionkey@0"
I20260926 06:24:12.121374 23747 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:24:12.122052 23761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:12.122280 23747 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:24:12.122411 23747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-1/wal
uuid: "ac72977b6c814241a3a55fb9b7e5c1e1"
format_stamp: "Formatted at 2026-09-26 06:24:12 on dist-test-slave-pd0t"
server_key: "bb7766d6a9a4129f5ea6da217509e3c9"
server_key_iv: "415caefb718e913226518f4bb446d4b9"
server_key_version: "encryptionkey@0"
I20260926 06:24:12.122481 23747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:12.145265 23747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:12.145535 23747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:12.145680 23747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:12.145893 23747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:24:12.146224 23747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:24:12.146276 23747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:12.146309 23747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:24:12.146337 23747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:12.151853 23747 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.130:38319
I20260926 06:24:12.151901 23874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.130:38319 every 8 connection(s)
I20260926 06:24:12.152233 23747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
I20260926 06:24:12.152618 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 23747
I20260926 06:24:12.152724 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance
I20260926 06:24:12.152983 4906 external_mini_cluster.cc:1428] Setting key 915d4cfc838e38b5748cf00b5f23c9e3
I20260926 06:24:12.154048 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.131:0
--local_ip_for_outbound_sockets=127.4.202.131
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:37959
--builtin_ntp_servers=127.4.202.148:39953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 06:24:12.158070 23595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:12.152988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:52017 (local address 127.4.202.190:37959)
0926 06:24:12.153061 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:12.153063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:12.153388 (+ 325us) server_negotiation.cc:407] Connection header received
0926 06:24:12.154010 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:12.154011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:12.154021 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:12.154063 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:12.155010 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.155477 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.156505 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.156818 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.156889 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:12.157740 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:12.157742 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:12.157743 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:12.157748 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:12.157767 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:12.157772 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:12.157834 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:12.157903 (+ 69us) server_negotiation.cc:299] Negotiation successful
0926 06:24:12.157939 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 06:24:12.158154 23877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:12.153019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37959 (local address 127.4.202.130:52017)
0926 06:24:12.153219 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:12.153260 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:12.153899 (+ 639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:12.154143 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:12.154150 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:12.154160 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:12.154862 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:12.154870 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.155612 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.155613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:12.156380 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:12.156382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.156904 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.156905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:12.157025 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:12.157547 (+ 522us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:24:12.157548 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:12.157574 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:24:12.157621 (+ 47us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:12.157834 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:12.157837 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:12.157839 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:24:12.157885 (+ 46us) client_negotiation.cc:240] Negotiation successful
0926 06:24:12.157960 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":97,"threads_started":1}
I20260926 06:24:12.158658 23875 heartbeater.cc:347] Connected to a master server at 127.4.202.190:37959
I20260926 06:24:12.158740 23875 heartbeater.cc:464] Registering TS with master...
I20260926 06:24:12.158933 23875 heartbeater.cc:511] Master 127.4.202.190:37959 requested a full tablet report, sending...
I20260926 06:24:12.159318 23556 ts_manager.cc:194] Registered new tserver with Master: ac72977b6c814241a3a55fb9b7e5c1e1 (127.4.202.130:38319)
I20260926 06:24:12.159852 23556 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.202.130:52017
W20260926 06:24:12.232409 23878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:12.232596 23878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:12.232617 23878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:12.233677 23878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:12.233713 23878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:12.233735 23878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.131
I20260926 06:24:12.234951 23878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:39953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:37959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.131
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:12.235112 23878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:12.235322 23878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:12.239794 23883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:12.239820 23884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:12.239781 23886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:12.239836 23878 server_base.cc:1034] running on GCE node
I20260926 06:24:12.240178 23878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:12.240384 23878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:12.241534 23878 hybrid_clock.cc:648] HybridClock initialized: now 1790403852241507 us; error 29 us; skew 500 ppm
I20260926 06:24:12.242731 23878 webserver.cc:460] Webserver started at http://127.4.202.131:34443/ using document root <none> and password file <none>
I20260926 06:24:12.242998 23878 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:12.243045 23878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:12.243186 23878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:12.244153 23878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-2/data/instance:
uuid: "4eaa8a28195b4a25aa5f0409e3ea24f7"
format_stamp: "Formatted at 2026-09-26 06:24:12 on dist-test-slave-pd0t"
server_key: "47068cd13a60d75cb9b7b6d22f909319"
server_key_iv: "c598e0d4239dcf4f52079d41b8eb864b"
server_key_version: "encryptionkey@0"
I20260926 06:24:12.244489 23878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance:
uuid: "4eaa8a28195b4a25aa5f0409e3ea24f7"
format_stamp: "Formatted at 2026-09-26 06:24:12 on dist-test-slave-pd0t"
server_key: "47068cd13a60d75cb9b7b6d22f909319"
server_key_iv: "c598e0d4239dcf4f52079d41b8eb864b"
server_key_version: "encryptionkey@0"
I20260926 06:24:12.245687 23878 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:24:12.246351 23892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:12.246536 23878 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:24:12.246652 23878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-2/wal
uuid: "4eaa8a28195b4a25aa5f0409e3ea24f7"
format_stamp: "Formatted at 2026-09-26 06:24:12 on dist-test-slave-pd0t"
server_key: "47068cd13a60d75cb9b7b6d22f909319"
server_key_iv: "c598e0d4239dcf4f52079d41b8eb864b"
server_key_version: "encryptionkey@0"
I20260926 06:24:12.246744 23878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:12.268560 23878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:12.268846 23878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:12.268998 23878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:12.269215 23878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:24:12.269528 23878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:24:12.269582 23878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:12.269616 23878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:24:12.269642 23878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:12.275326 23878 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.131:45175
I20260926 06:24:12.275375 24005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.131:45175 every 8 connection(s)
I20260926 06:24:12.275668 23878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
I20260926 06:24:12.279240 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 23878
I20260926 06:24:12.279366 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance
I20260926 06:24:12.279577 4906 external_mini_cluster.cc:1428] Setting key 6d2ca6fb104afd76939d9cf805bab933
I20260926 06:24:12.280510 23595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:12.276316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:47151 (local address 127.4.202.190:37959)
0926 06:24:12.276380 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:12.276382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:12.276613 (+ 231us) server_negotiation.cc:407] Connection header received
0926 06:24:12.277066 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:12.277066 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:12.277075 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:12.277130 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:12.277710 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.278165 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.279066 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.279240 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.279289 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:12.280208 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:12.280209 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:12.280210 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:12.280214 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:12.280230 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:12.280234 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:12.280287 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:12.280388 (+ 101us) server_negotiation.cc:299] Negotiation successful
0926 06:24:12.280415 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 06:24:12.280664 24008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:12.276295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:37959 (local address 127.4.202.131:47151)
0926 06:24:12.276489 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:12.276527 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:12.276988 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:12.277178 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:12.277181 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:12.277189 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:12.277647 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:12.277651 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.278243 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.278244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:12.278998 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:12.279000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.279328 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.279330 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:12.279489 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:12.280061 (+ 572us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:24:12.280061 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:12.280096 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:24:12.280121 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:12.280295 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:12.280300 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:12.280302 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 06:24:12.280378 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 06:24:12.280474 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":106,"threads_started":1}
I20260926 06:24:12.281107 24006 heartbeater.cc:347] Connected to a master server at 127.4.202.190:37959
I20260926 06:24:12.281188 24006 heartbeater.cc:464] Registering TS with master...
I20260926 06:24:12.281347 24006 heartbeater.cc:511] Master 127.4.202.190:37959 requested a full tablet report, sending...
I20260926 06:24:12.281673 23556 ts_manager.cc:194] Registered new tserver with Master: 4eaa8a28195b4a25aa5f0409e3ea24f7 (127.4.202.131:45175)
I20260926 06:24:12.281988 23556 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.202.131:47151
I20260926 06:24:12.291220 4906 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:24:12.295275 23595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:12.292507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53368 (local address 127.4.202.190:37959)
0926 06:24:12.292571 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:12.292577 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:12.292612 (+ 35us) server_negotiation.cc:407] Connection header received
0926 06:24:12.292662 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:12.292663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:12.292671 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:12.292715 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:12.292859 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.293295 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.294076 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.294247 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.294296 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:12.295010 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:12.295011 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:12.295012 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:12.295015 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:12.295027 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:12.295031 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:12.295098 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:12.295157 (+ 59us) server_negotiation.cc:299] Negotiation successful
0926 06:24:12.295166 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 06:24:12.296595 23556 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53368:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:24:12.298074 23556 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:24:12.304149 24019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:12.300343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60572 (local address 127.4.202.131:45175)
0926 06:24:12.300507 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:12.300511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:12.300518 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:24:12.300545 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:12.300546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:12.300557 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:12.300652 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:24:12.300930 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.301711 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.303094 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.303743 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.303849 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:12.303930 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:12.304021 (+ 91us) server_negotiation.cc:299] Negotiation successful
0926 06:24:12.304054 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":81,"threads_started":1}
I20260926 06:24:12.304152 24016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:12.299998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:45175 (local address 127.0.0.1:60572)
0926 06:24:12.300244 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:12.300294 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:12.300359 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:12.300680 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:12.300683 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:12.300685 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:24:12.300847 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:12.300850 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.301795 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.301796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:12.303008 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:12.303010 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.303853 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.303854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:12.303926 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:12.303939 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 06:24:12.304008 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 06:24:12.304054 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":78,"threads_started":1}
I20260926 06:24:12.304384 24015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:12.299999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:38319 (local address 127.0.0.1:37142)
0926 06:24:12.300529 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:12.300537 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:12.300588 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:12.300796 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:12.300797 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:12.300799 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:24:12.300850 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:12.300852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.302141 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.302142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:12.303168 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:12.303170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.304146 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.304147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:12.304209 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:12.304213 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:24:12.304267 (+ 54us) client_negotiation.cc:240] Negotiation successful
0926 06:24:12.304306 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"mutex_wait_us":2,"thread_start_us":75,"threads_started":1}
I20260926 06:24:12.304601 24014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:12.300004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:33561 (local address 127.0.0.1:39620)
0926 06:24:12.300660 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:12.300668 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:12.300722 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:12.301099 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:12.301100 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:12.301102 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:24:12.301135 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:12.301136 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.302350 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.302351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:12.303409 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:12.303412 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.304379 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.304381 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:12.304453 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:12.304456 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:24:12.304509 (+ 53us) client_negotiation.cc:240] Negotiation successful
0926 06:24:12.304540 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":92,"threads_started":1}
I20260926 06:24:12.304620 24017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:12.300175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37142 (local address 127.4.202.130:38319)
0926 06:24:12.300351 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:12.300355 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:12.300633 (+ 278us) server_negotiation.cc:407] Connection header received
0926 06:24:12.300674 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:12.300676 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:12.300691 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:12.300783 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:24:12.300924 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.302031 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.303261 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.303948 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.304016 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:12.304092 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:12.304496 (+ 404us) server_negotiation.cc:299] Negotiation successful
0926 06:24:12.304539 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":84,"threads_started":1}
I20260926 06:24:12.304893 24018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:12.300234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39620 (local address 127.4.202.129:33561)
0926 06:24:12.300414 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:12.300418 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:12.300796 (+ 378us) server_negotiation.cc:407] Connection header received
0926 06:24:12.300916 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:12.300918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:12.300955 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:12.301031 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:24:12.301217 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.302264 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.303500 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.304213 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.304275 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:12.304348 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:12.304771 (+ 423us) server_negotiation.cc:299] Negotiation successful
0926 06:24:12.304810 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":70,"threads_started":1}
I20260926 06:24:12.305133 23940 tablet_service.cc:1467] Processing CreateTablet for tablet e9974d1d5b1047b092e1eeb924cb6e73 (DEFAULT_TABLE table=test-table [id=08c5145b6e0047f6af1c1971191f9d8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:24:12.305186 23809 tablet_service.cc:1467] Processing CreateTablet for tablet e9974d1d5b1047b092e1eeb924cb6e73 (DEFAULT_TABLE table=test-table [id=08c5145b6e0047f6af1c1971191f9d8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:24:12.305474 23809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9974d1d5b1047b092e1eeb924cb6e73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:12.305474 23940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9974d1d5b1047b092e1eeb924cb6e73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:12.305502 23678 tablet_service.cc:1467] Processing CreateTablet for tablet e9974d1d5b1047b092e1eeb924cb6e73 (DEFAULT_TABLE table=test-table [id=08c5145b6e0047f6af1c1971191f9d8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:24:12.305836 23678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9974d1d5b1047b092e1eeb924cb6e73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:12.308282 24020 tablet_bootstrap.cc:492] T e9974d1d5b1047b092e1eeb924cb6e73 P 81ef1594c8c84c609d92a82300dd90f4: Bootstrap starting.
I20260926 06:24:12.308632 24022 tablet_bootstrap.cc:492] T e9974d1d5b1047b092e1eeb924cb6e73 P ac72977b6c814241a3a55fb9b7e5c1e1: Bootstrap starting.
I20260926 06:24:12.309031 24020 tablet_bootstrap.cc:654] T e9974d1d5b1047b092e1eeb924cb6e73 P 81ef1594c8c84c609d92a82300dd90f4: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:12.309275 24022 tablet_bootstrap.cc:654] T e9974d1d5b1047b092e1eeb924cb6e73 P ac72977b6c814241a3a55fb9b7e5c1e1: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:12.309356 24020 log.cc:824] T e9974d1d5b1047b092e1eeb924cb6e73 P 81ef1594c8c84c609d92a82300dd90f4: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:12.309576 24022 log.cc:824] T e9974d1d5b1047b092e1eeb924cb6e73 P ac72977b6c814241a3a55fb9b7e5c1e1: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:12.309903 24021 tablet_bootstrap.cc:492] T e9974d1d5b1047b092e1eeb924cb6e73 P 4eaa8a28195b4a25aa5f0409e3ea24f7: Bootstrap starting.
I20260926 06:24:12.310177 24020 tablet_bootstrap.cc:492] T e9974d1d5b1047b092e1eeb924cb6e73 P 81ef1594c8c84c609d92a82300dd90f4: No bootstrap required, opened a new log
I20260926 06:24:12.310258 24020 ts_tablet_manager.cc:1397] T e9974d1d5b1047b092e1eeb924cb6e73 P 81ef1594c8c84c609d92a82300dd90f4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:24:12.310491 24021 tablet_bootstrap.cc:654] T e9974d1d5b1047b092e1eeb924cb6e73 P 4eaa8a28195b4a25aa5f0409e3ea24f7: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:12.310781 24021 log.cc:824] T e9974d1d5b1047b092e1eeb924cb6e73 P 4eaa8a28195b4a25aa5f0409e3ea24f7: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:12.311271 24022 tablet_bootstrap.cc:492] T e9974d1d5b1047b092e1eeb924cb6e73 P ac72977b6c814241a3a55fb9b7e5c1e1: No bootstrap required, opened a new log
I20260926 06:24:12.311352 24022 ts_tablet_manager.cc:1397] T e9974d1d5b1047b092e1eeb924cb6e73 P ac72977b6c814241a3a55fb9b7e5c1e1: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260926 06:24:12.311474 24021 tablet_bootstrap.cc:492] T e9974d1d5b1047b092e1eeb924cb6e73 P 4eaa8a28195b4a25aa5f0409e3ea24f7: No bootstrap required, opened a new log
I20260926 06:24:12.311614 24021 ts_tablet_manager.cc:1397] T e9974d1d5b1047b092e1eeb924cb6e73 P 4eaa8a28195b4a25aa5f0409e3ea24f7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 06:24:12.312906 24020 raft_consensus.cc:348] T e9974d1d5b1047b092e1eeb924cb6e73 P 81ef1594c8c84c609d92a82300dd90f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eaa8a28195b4a25aa5f0409e3ea24f7" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 45175 } } peers { permanent_uuid: "ac72977b6c814241a3a55fb9b7e5c1e1" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38319 } } peers { permanent_uuid: "81ef1594c8c84c609d92a82300dd90f4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33561 } }
I20260926 06:24:12.313067 24020 raft_consensus.cc:374] T e9974d1d5b1047b092e1eeb924cb6e73 P 81ef1594c8c84c609d92a82300dd90f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:12.313103 24020 raft_consensus.cc:729] T e9974d1d5b1047b092e1eeb924cb6e73 P 81ef1594c8c84c609d92a82300dd90f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81ef1594c8c84c609d92a82300dd90f4, State: Initialized, Role: FOLLOWER
I20260926 06:24:12.313246 24020 consensus_queue.cc:260] T e9974d1d5b1047b092e1eeb924cb6e73 P 81ef1594c8c84c609d92a82300dd90f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eaa8a28195b4a25aa5f0409e3ea24f7" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 45175 } } peers { permanent_uuid: "ac72977b6c814241a3a55fb9b7e5c1e1" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38319 } } peers { permanent_uuid: "81ef1594c8c84c609d92a82300dd90f4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33561 } }
I20260926 06:24:12.313264 24022 raft_consensus.cc:348] T e9974d1d5b1047b092e1eeb924cb6e73 P ac72977b6c814241a3a55fb9b7e5c1e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eaa8a28195b4a25aa5f0409e3ea24f7" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 45175 } } peers { permanent_uuid: "ac72977b6c814241a3a55fb9b7e5c1e1" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38319 } } peers { permanent_uuid: "81ef1594c8c84c609d92a82300dd90f4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33561 } }
I20260926 06:24:12.313423 24022 raft_consensus.cc:374] T e9974d1d5b1047b092e1eeb924cb6e73 P ac72977b6c814241a3a55fb9b7e5c1e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:12.313465 24022 raft_consensus.cc:729] T e9974d1d5b1047b092e1eeb924cb6e73 P ac72977b6c814241a3a55fb9b7e5c1e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac72977b6c814241a3a55fb9b7e5c1e1, State: Initialized, Role: FOLLOWER
I20260926 06:24:12.313484 24020 ts_tablet_manager.cc:1428] T e9974d1d5b1047b092e1eeb924cb6e73 P 81ef1594c8c84c609d92a82300dd90f4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 06:24:12.313596 23744 heartbeater.cc:503] Master 127.4.202.190:37959 was elected leader, sending a full tablet report...
I20260926 06:24:12.313829 24022 consensus_queue.cc:260] T e9974d1d5b1047b092e1eeb924cb6e73 P ac72977b6c814241a3a55fb9b7e5c1e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eaa8a28195b4a25aa5f0409e3ea24f7" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 45175 } } peers { permanent_uuid: "ac72977b6c814241a3a55fb9b7e5c1e1" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38319 } } peers { permanent_uuid: "81ef1594c8c84c609d92a82300dd90f4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33561 } }
I20260926 06:24:12.314109 23875 heartbeater.cc:503] Master 127.4.202.190:37959 was elected leader, sending a full tablet report...
I20260926 06:24:12.314091 24022 ts_tablet_manager.cc:1428] T e9974d1d5b1047b092e1eeb924cb6e73 P ac72977b6c814241a3a55fb9b7e5c1e1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 06:24:12.314095 24021 raft_consensus.cc:348] T e9974d1d5b1047b092e1eeb924cb6e73 P 4eaa8a28195b4a25aa5f0409e3ea24f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eaa8a28195b4a25aa5f0409e3ea24f7" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 45175 } } peers { permanent_uuid: "ac72977b6c814241a3a55fb9b7e5c1e1" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38319 } } peers { permanent_uuid: "81ef1594c8c84c609d92a82300dd90f4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33561 } }
I20260926 06:24:12.314251 24021 raft_consensus.cc:374] T e9974d1d5b1047b092e1eeb924cb6e73 P 4eaa8a28195b4a25aa5f0409e3ea24f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:12.314301 24021 raft_consensus.cc:729] T e9974d1d5b1047b092e1eeb924cb6e73 P 4eaa8a28195b4a25aa5f0409e3ea24f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4eaa8a28195b4a25aa5f0409e3ea24f7, State: Initialized, Role: FOLLOWER
I20260926 06:24:12.314419 24021 consensus_queue.cc:260] T e9974d1d5b1047b092e1eeb924cb6e73 P 4eaa8a28195b4a25aa5f0409e3ea24f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eaa8a28195b4a25aa5f0409e3ea24f7" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 45175 } } peers { permanent_uuid: "ac72977b6c814241a3a55fb9b7e5c1e1" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38319 } } peers { permanent_uuid: "81ef1594c8c84c609d92a82300dd90f4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33561 } }
I20260926 06:24:12.314693 24021 ts_tablet_manager.cc:1428] T e9974d1d5b1047b092e1eeb924cb6e73 P 4eaa8a28195b4a25aa5f0409e3ea24f7: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 06:24:12.314724 24006 heartbeater.cc:503] Master 127.4.202.190:37959 was elected leader, sending a full tablet report...
I20260926 06:24:12.319723 24028 raft_consensus.cc:482] T e9974d1d5b1047b092e1eeb924cb6e73 P 4eaa8a28195b4a25aa5f0409e3ea24f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:24:12.319792 24028 raft_consensus.cc:504] T e9974d1d5b1047b092e1eeb924cb6e73 P 4eaa8a28195b4a25aa5f0409e3ea24f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eaa8a28195b4a25aa5f0409e3ea24f7" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 45175 } } peers { permanent_uuid: "ac72977b6c814241a3a55fb9b7e5c1e1" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38319 } } peers { permanent_uuid: "81ef1594c8c84c609d92a82300dd90f4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33561 } }
I20260926 06:24:12.320098 24028 leader_election.cc:288] T e9974d1d5b1047b092e1eeb924cb6e73 P 4eaa8a28195b4a25aa5f0409e3ea24f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ac72977b6c814241a3a55fb9b7e5c1e1 (127.4.202.130:38319), 81ef1594c8c84c609d92a82300dd90f4 (127.4.202.129:33561)
I20260926 06:24:12.322657 24017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:12.320378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:40847 (local address 127.4.202.130:38319)
0926 06:24:12.320413 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:12.320416 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:12.320424 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:24:12.320452 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:12.320452 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:12.320459 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:12.320538 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:24:12.320672 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.321097 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.321960 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.322302 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.322390 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:12.322397 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:12.322547 (+ 150us) server_negotiation.cc:299] Negotiation successful
0926 06:24:12.322605 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 06:24:12.322659 24008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:12.320240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:38319 (local address 127.4.202.131:40847)
0926 06:24:12.320286 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:12.320295 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:12.320328 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:12.320541 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:12.320542 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:12.320544 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:24:12.320595 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:12.320596 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.321172 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.321172 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:12.321854 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:12.321856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.322392 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.322392 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:12.322465 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:12.322468 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:24:12.322538 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 06:24:12.322596 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":4}
I20260926 06:24:12.322921 24018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:12.320375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:48501 (local address 127.4.202.129:33561)
0926 06:24:12.320431 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:12.320433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:12.320529 (+ 96us) server_negotiation.cc:407] Connection header received
0926 06:24:12.320587 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:12.320588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:12.320597 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:12.320656 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:24:12.320800 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.321223 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.322083 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.322424 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.322529 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:12.322535 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:12.322820 (+ 285us) server_negotiation.cc:299] Negotiation successful
0926 06:24:12.322863 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 06:24:12.322953 24029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:12.320288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:33561 (local address 127.4.202.131:48501)
0926 06:24:12.320460 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:12.320469 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:12.320541 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:12.320659 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:12.320660 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:12.320661 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:24:12.320722 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:12.320723 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.321300 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.321301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:12.321996 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:12.321997 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.322530 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.322531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:12.322635 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:12.322638 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:24:12.322850 (+ 212us) client_negotiation.cc:240] Negotiation successful
0926 06:24:12.322883 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":79,"threads_started":1}
I20260926 06:24:12.322913 23829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9974d1d5b1047b092e1eeb924cb6e73" candidate_uuid: "4eaa8a28195b4a25aa5f0409e3ea24f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac72977b6c814241a3a55fb9b7e5c1e1" is_pre_election: true
I20260926 06:24:12.323055 23829 raft_consensus.cc:2454] T e9974d1d5b1047b092e1eeb924cb6e73 P ac72977b6c814241a3a55fb9b7e5c1e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4eaa8a28195b4a25aa5f0409e3ea24f7 in term 0.
I20260926 06:24:12.323210 23698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9974d1d5b1047b092e1eeb924cb6e73" candidate_uuid: "4eaa8a28195b4a25aa5f0409e3ea24f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81ef1594c8c84c609d92a82300dd90f4" is_pre_election: true
I20260926 06:24:12.323271 23893 leader_election.cc:302] T e9974d1d5b1047b092e1eeb924cb6e73 P 4eaa8a28195b4a25aa5f0409e3ea24f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4eaa8a28195b4a25aa5f0409e3ea24f7, ac72977b6c814241a3a55fb9b7e5c1e1; no voters:
I20260926 06:24:12.323328 23698 raft_consensus.cc:2454] T e9974d1d5b1047b092e1eeb924cb6e73 P 81ef1594c8c84c609d92a82300dd90f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4eaa8a28195b4a25aa5f0409e3ea24f7 in term 0.
I20260926 06:24:12.323366 24028 raft_consensus.cc:2781] T e9974d1d5b1047b092e1eeb924cb6e73 P 4eaa8a28195b4a25aa5f0409e3ea24f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:24:12.323429 24028 raft_consensus.cc:482] T e9974d1d5b1047b092e1eeb924cb6e73 P 4eaa8a28195b4a25aa5f0409e3ea24f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:24:12.323462 24028 raft_consensus.cc:3037] T e9974d1d5b1047b092e1eeb924cb6e73 P 4eaa8a28195b4a25aa5f0409e3ea24f7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:12.324038 24028 raft_consensus.cc:504] T e9974d1d5b1047b092e1eeb924cb6e73 P 4eaa8a28195b4a25aa5f0409e3ea24f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eaa8a28195b4a25aa5f0409e3ea24f7" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 45175 } } peers { permanent_uuid: "ac72977b6c814241a3a55fb9b7e5c1e1" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38319 } } peers { permanent_uuid: "81ef1594c8c84c609d92a82300dd90f4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33561 } }
I20260926 06:24:12.324175 24028 leader_election.cc:288] T e9974d1d5b1047b092e1eeb924cb6e73 P 4eaa8a28195b4a25aa5f0409e3ea24f7 [CANDIDATE]: Term 1 election: Requested vote from peers ac72977b6c814241a3a55fb9b7e5c1e1 (127.4.202.130:38319), 81ef1594c8c84c609d92a82300dd90f4 (127.4.202.129:33561)
I20260926 06:24:12.324299 23829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9974d1d5b1047b092e1eeb924cb6e73" candidate_uuid: "4eaa8a28195b4a25aa5f0409e3ea24f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac72977b6c814241a3a55fb9b7e5c1e1"
I20260926 06:24:12.324361 23698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9974d1d5b1047b092e1eeb924cb6e73" candidate_uuid: "4eaa8a28195b4a25aa5f0409e3ea24f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81ef1594c8c84c609d92a82300dd90f4"
I20260926 06:24:12.324368 23829 raft_consensus.cc:3037] T e9974d1d5b1047b092e1eeb924cb6e73 P ac72977b6c814241a3a55fb9b7e5c1e1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:12.324455 23698 raft_consensus.cc:3037] T e9974d1d5b1047b092e1eeb924cb6e73 P 81ef1594c8c84c609d92a82300dd90f4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:12.325147 23698 raft_consensus.cc:2454] T e9974d1d5b1047b092e1eeb924cb6e73 P 81ef1594c8c84c609d92a82300dd90f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4eaa8a28195b4a25aa5f0409e3ea24f7 in term 1.
I20260926 06:24:12.325150 23829 raft_consensus.cc:2454] T e9974d1d5b1047b092e1eeb924cb6e73 P ac72977b6c814241a3a55fb9b7e5c1e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4eaa8a28195b4a25aa5f0409e3ea24f7 in term 1.
I20260926 06:24:12.325322 23895 leader_election.cc:302] T e9974d1d5b1047b092e1eeb924cb6e73 P 4eaa8a28195b4a25aa5f0409e3ea24f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4eaa8a28195b4a25aa5f0409e3ea24f7, 81ef1594c8c84c609d92a82300dd90f4; no voters:
I20260926 06:24:12.325457 24028 raft_consensus.cc:2781] T e9974d1d5b1047b092e1eeb924cb6e73 P 4eaa8a28195b4a25aa5f0409e3ea24f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:24:12.325634 24028 raft_consensus.cc:686] T e9974d1d5b1047b092e1eeb924cb6e73 P 4eaa8a28195b4a25aa5f0409e3ea24f7 [term 1 LEADER]: Becoming Leader. State: Replica: 4eaa8a28195b4a25aa5f0409e3ea24f7, State: Running, Role: LEADER
I20260926 06:24:12.325736 24028 consensus_queue.cc:237] T e9974d1d5b1047b092e1eeb924cb6e73 P 4eaa8a28195b4a25aa5f0409e3ea24f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eaa8a28195b4a25aa5f0409e3ea24f7" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 45175 } } peers { permanent_uuid: "ac72977b6c814241a3a55fb9b7e5c1e1" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38319 } } peers { permanent_uuid: "81ef1594c8c84c609d92a82300dd90f4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33561 } }
I20260926 06:24:12.326367 23556 catalog_manager.cc:5495] T e9974d1d5b1047b092e1eeb924cb6e73 P 4eaa8a28195b4a25aa5f0409e3ea24f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4eaa8a28195b4a25aa5f0409e3ea24f7 (127.4.202.131). New cstate: current_term: 1 leader_uuid: "4eaa8a28195b4a25aa5f0409e3ea24f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eaa8a28195b4a25aa5f0409e3ea24f7" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 45175 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac72977b6c814241a3a55fb9b7e5c1e1" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81ef1594c8c84c609d92a82300dd90f4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33561 } health_report { overall_health: UNKNOWN } } }
I20260926 06:24:12.334390 24019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:12.332149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60586 (local address 127.4.202.131:45175)
0926 06:24:12.332209 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:12.332211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:12.332218 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:24:12.332313 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:12.332314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:12.332330 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:12.332387 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:24:12.332500 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.332909 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.333650 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:12.333845 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:12.333922 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:12.334074 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:24:12.334162 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:24:12.334257 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:12.334303 (+ 46us) server_negotiation.cc:299] Negotiation successful
0926 06:24:12.334310 (+ 7us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 06:24:12.335906 23829 raft_consensus.cc:1261] T e9974d1d5b1047b092e1eeb924cb6e73 P ac72977b6c814241a3a55fb9b7e5c1e1 [term 1 FOLLOWER]: Refusing update from remote peer 4eaa8a28195b4a25aa5f0409e3ea24f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:24:12.335940 23698 raft_consensus.cc:1261] T e9974d1d5b1047b092e1eeb924cb6e73 P 81ef1594c8c84c609d92a82300dd90f4 [term 1 FOLLOWER]: Refusing update from remote peer 4eaa8a28195b4a25aa5f0409e3ea24f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:24:12.336217 24030 consensus_queue.cc:1035] T e9974d1d5b1047b092e1eeb924cb6e73 P 4eaa8a28195b4a25aa5f0409e3ea24f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81ef1594c8c84c609d92a82300dd90f4" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 33561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:24:12.336318 24028 consensus_queue.cc:1035] T e9974d1d5b1047b092e1eeb924cb6e73 P 4eaa8a28195b4a25aa5f0409e3ea24f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac72977b6c814241a3a55fb9b7e5c1e1" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 38319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:24:12.338321 24033 mvcc.cc:205] Tried to move back new op lower bound from 7333494179165646848 to 7333494179126792192. Current Snapshot: MvccSnapshot[applied={T|T < 7333494179165646848}]
I20260926 06:24:12.338394 24035 mvcc.cc:205] Tried to move back new op lower bound from 7333494179165646848 to 7333494179126792192. Current Snapshot: MvccSnapshot[applied={T|T < 7333494179165646848}]
I20260926 06:24:12.338479 24034 mvcc.cc:205] Tried to move back new op lower bound from 7333494179165646848 to 7333494179126792192. Current Snapshot: MvccSnapshot[applied={T|T < 7333494179165646848}]
I20260926 06:24:12.340363 23555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:24:12.340473 23555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:24:12.341166 23555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aa11d9198a004307aff5c8252f206233: Sending DeleteTablet for 3 replicas of tablet e9974d1d5b1047b092e1eeb924cb6e73
I20260926 06:24:12.341457 23940 tablet_service.cc:1514] Processing DeleteTablet for tablet e9974d1d5b1047b092e1eeb924cb6e73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:24:12 UTC) from {username='slave'} at 127.0.0.1:60572
I20260926 06:24:12.341467 23809 tablet_service.cc:1514] Processing DeleteTablet for tablet e9974d1d5b1047b092e1eeb924cb6e73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:24:12 UTC) from {username='slave'} at 127.0.0.1:37142
I20260926 06:24:12.341542 23678 tablet_service.cc:1514] Processing DeleteTablet for tablet e9974d1d5b1047b092e1eeb924cb6e73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:24:12 UTC) from {username='slave'} at 127.0.0.1:39620
I20260926 06:24:12.341703 24043 tablet_replica.cc:330] T e9974d1d5b1047b092e1eeb924cb6e73 P 4eaa8a28195b4a25aa5f0409e3ea24f7: stopping tablet replica
I20260926 06:24:12.341737 24045 tablet_replica.cc:330] T e9974d1d5b1047b092e1eeb924cb6e73 P 81ef1594c8c84c609d92a82300dd90f4: stopping tablet replica
I20260926 06:24:12.341853 24044 tablet_replica.cc:330] T e9974d1d5b1047b092e1eeb924cb6e73 P ac72977b6c814241a3a55fb9b7e5c1e1: stopping tablet replica
I20260926 06:24:12.341886 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 23616
I20260926 06:24:12.342666 24044 raft_consensus.cc:2229] T e9974d1d5b1047b092e1eeb924cb6e73 P ac72977b6c814241a3a55fb9b7e5c1e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:24:12.342797 24043 raft_consensus.cc:2229] T e9974d1d5b1047b092e1eeb924cb6e73 P 4eaa8a28195b4a25aa5f0409e3ea24f7 [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:24:12.342936 24043 raft_consensus.cc:2258] T e9974d1d5b1047b092e1eeb924cb6e73 P 4eaa8a28195b4a25aa5f0409e3ea24f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:24:12.343070 24044 raft_consensus.cc:2258] T e9974d1d5b1047b092e1eeb924cb6e73 P ac72977b6c814241a3a55fb9b7e5c1e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:24:12.343585 24043 ts_tablet_manager.cc:1905] T e9974d1d5b1047b092e1eeb924cb6e73 P 4eaa8a28195b4a25aa5f0409e3ea24f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:24:12.344805 24044 ts_tablet_manager.cc:1905] T e9974d1d5b1047b092e1eeb924cb6e73 P ac72977b6c814241a3a55fb9b7e5c1e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:24:12.345086 24043 ts_tablet_manager.cc:1918] T e9974d1d5b1047b092e1eeb924cb6e73 P 4eaa8a28195b4a25aa5f0409e3ea24f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:24:12.345207 24043 log.cc:1196] T e9974d1d5b1047b092e1eeb924cb6e73 P 4eaa8a28195b4a25aa5f0409e3ea24f7: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-2/wal/wals/e9974d1d5b1047b092e1eeb924cb6e73
I20260926 06:24:12.345516 24043 ts_tablet_manager.cc:1939] T e9974d1d5b1047b092e1eeb924cb6e73 P 4eaa8a28195b4a25aa5f0409e3ea24f7: Deleting consensus metadata
I20260926 06:24:12.346313 24044 ts_tablet_manager.cc:1918] T e9974d1d5b1047b092e1eeb924cb6e73 P ac72977b6c814241a3a55fb9b7e5c1e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:24:12.346395 24044 log.cc:1196] T e9974d1d5b1047b092e1eeb924cb6e73 P ac72977b6c814241a3a55fb9b7e5c1e1: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790403591640726-4906-0/minicluster-data/ts-1/wal/wals/e9974d1d5b1047b092e1eeb924cb6e73
I20260926 06:24:12.346688 24044 ts_tablet_manager.cc:1939] T e9974d1d5b1047b092e1eeb924cb6e73 P ac72977b6c814241a3a55fb9b7e5c1e1: Deleting consensus metadata
I20260926 06:24:12.347043 23540 catalog_manager.cc:4841] TS ac72977b6c814241a3a55fb9b7e5c1e1 (127.4.202.130:38319): tablet e9974d1d5b1047b092e1eeb924cb6e73 (table test-table [id=08c5145b6e0047f6af1c1971191f9d8e]) successfully deleted
I20260926 06:24:12.347858 23541 catalog_manager.cc:4841] TS 4eaa8a28195b4a25aa5f0409e3ea24f7 (127.4.202.131:45175): tablet e9974d1d5b1047b092e1eeb924cb6e73 (table test-table [id=08c5145b6e0047f6af1c1971191f9d8e]) successfully deleted
I20260926 06:24:12.349418 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 23747
W20260926 06:24:12.349898 23542 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.202.129:33561 (error 108)
I20260926 06:24:12.351647 24014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:12.351509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:33561 (local address 127.0.0.1:39632)
0926 06:24:12.351570 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:12.351612 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.129:33561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260926 06:24:12.351778 23542 catalog_manager.cc:4584] TS 81ef1594c8c84c609d92a82300dd90f4 (127.4.202.129:33561): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e9974d1d5b1047b092e1eeb924cb6e73: Network error: Client connection negotiation failed: client connection to 127.4.202.129:33561: connect: Connection refused (error 111)
I20260926 06:24:12.354532 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 23878
I20260926 06:24:12.359625 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 23525
2026-09-26T06:24:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (645 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:24:12 dist-test-slave-pd0t krb5kdc[24049](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:24:12 dist-test-slave-pd0t krb5kdc[24049](info): set up 2 sockets
Sep 26 06:24:12 dist-test-slave-pd0t krb5kdc[24049](info): commencing operation
krb5kdc: starting...
W20260926 06:24:13.370291 4906 mini_kdc.cc:121] Time spent starting KDC: real 0.992s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:24:13 dist-test-slave-pd0t krb5kdc[24049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:24:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:24:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.202.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
I20260926 06:24:13.481184 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:40485
--webserver_interface=127.4.202.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:37305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:40485
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:24:13.559090 24062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:13.559303 24062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:13.559338 24062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:13.560446 24062 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:24:13.560509 24062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:13.560531 24062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:13.560551 24062 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:24:13.560581 24062 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:24:13.562003 24062 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:37305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:40485
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:40485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:13.562251 24062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:13.562451 24062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:13.566990 24067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:13.567001 24068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:13.567001 24070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:13.567198 24062 server_base.cc:1034] running on GCE node
I20260926 06:24:13.567452 24062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:13.567768 24062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:13.568936 24062 hybrid_clock.cc:648] HybridClock initialized: now 1790403853568923 us; error 31 us; skew 500 ppm
Sep 26 06:24:13 dist-test-slave-pd0t krb5kdc[24049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403853, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:13.571209 24062 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:24:13.571596 24062 webserver.cc:460] Webserver started at http://127.4.202.190:35315/ using document root <none> and password file <none>
I20260926 06:24:13.571817 24062 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:13.571897 24062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:13.572054 24062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:13.573055 24062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "1fbe6a1abebc438bacde58b25c3a0b40"
format_stamp: "Formatted at 2026-09-26 06:24:13 on dist-test-slave-pd0t"
server_key: "1827a5be7865ad6fa799d6df4e79804a"
server_key_iv: "9c31f34539ee0fee845aad6819d992bb"
server_key_version: "encryptionkey@0"
I20260926 06:24:13.573398 24062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "1fbe6a1abebc438bacde58b25c3a0b40"
format_stamp: "Formatted at 2026-09-26 06:24:13 on dist-test-slave-pd0t"
server_key: "1827a5be7865ad6fa799d6df4e79804a"
server_key_iv: "9c31f34539ee0fee845aad6819d992bb"
server_key_version: "encryptionkey@0"
I20260926 06:24:13.574709 24062 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:24:13.575423 24077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:13.575604 24062 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:24:13.575712 24062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "1fbe6a1abebc438bacde58b25c3a0b40"
format_stamp: "Formatted at 2026-09-26 06:24:13 on dist-test-slave-pd0t"
server_key: "1827a5be7865ad6fa799d6df4e79804a"
server_key_iv: "9c31f34539ee0fee845aad6819d992bb"
server_key_version: "encryptionkey@0"
I20260926 06:24:13.575810 24062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:13.588711 24062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:13.590366 24062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:13.590521 24062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:13.594118 24062 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:40485
I20260926 06:24:13.594172 24129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:40485 every 8 connection(s)
I20260926 06:24:13.594509 24062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:24:13.595294 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 24062
I20260926 06:24:13.595412 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:24:13.595412 24130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:13.595628 4906 external_mini_cluster.cc:1428] Setting key 320d8f94524f87458db3fcf56453aa60
I20260926 06:24:13.598377 24130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fbe6a1abebc438bacde58b25c3a0b40: Bootstrap starting.
I20260926 06:24:13.599077 24130 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1fbe6a1abebc438bacde58b25c3a0b40: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:13.599357 24130 log.cc:824] T 00000000000000000000000000000000 P 1fbe6a1abebc438bacde58b25c3a0b40: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:13.599990 24130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fbe6a1abebc438bacde58b25c3a0b40: No bootstrap required, opened a new log
I20260926 06:24:13.601708 24130 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1fbe6a1abebc438bacde58b25c3a0b40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fbe6a1abebc438bacde58b25c3a0b40" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 40485 } }
I20260926 06:24:13.601828 24130 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1fbe6a1abebc438bacde58b25c3a0b40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:13.601851 24130 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1fbe6a1abebc438bacde58b25c3a0b40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fbe6a1abebc438bacde58b25c3a0b40, State: Initialized, Role: FOLLOWER
I20260926 06:24:13.601991 24130 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fbe6a1abebc438bacde58b25c3a0b40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fbe6a1abebc438bacde58b25c3a0b40" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 40485 } }
I20260926 06:24:13.602051 24130 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1fbe6a1abebc438bacde58b25c3a0b40 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:24:13.602072 24130 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1fbe6a1abebc438bacde58b25c3a0b40 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:24:13.602141 24130 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1fbe6a1abebc438bacde58b25c3a0b40 [term 0 FOLLOWER]: Advancing to term 1
Sep 26 06:24:13 dist-test-slave-pd0t krb5kdc[24049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:24:13.602700 24130 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1fbe6a1abebc438bacde58b25c3a0b40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fbe6a1abebc438bacde58b25c3a0b40" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 40485 } }
I20260926 06:24:13.602783 24130 leader_election.cc:302] T 00000000000000000000000000000000 P 1fbe6a1abebc438bacde58b25c3a0b40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1fbe6a1abebc438bacde58b25c3a0b40; no voters:
I20260926 06:24:13.602969 24130 leader_election.cc:288] T 00000000000000000000000000000000 P 1fbe6a1abebc438bacde58b25c3a0b40 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:24:13.603076 24135 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1fbe6a1abebc438bacde58b25c3a0b40 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:24:13.603237 24130 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fbe6a1abebc438bacde58b25c3a0b40 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:24:13.603554 24135 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1fbe6a1abebc438bacde58b25c3a0b40 [term 1 LEADER]: Becoming Leader. State: Replica: 1fbe6a1abebc438bacde58b25c3a0b40, State: Running, Role: LEADER
I20260926 06:24:13.603669 24135 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1fbe6a1abebc438bacde58b25c3a0b40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fbe6a1abebc438bacde58b25c3a0b40" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 40485 } }
I20260926 06:24:13.605451 24136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fbe6a1abebc438bacde58b25c3a0b40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1fbe6a1abebc438bacde58b25c3a0b40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fbe6a1abebc438bacde58b25c3a0b40" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 40485 } } }
I20260926 06:24:13.605546 24136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fbe6a1abebc438bacde58b25c3a0b40 [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:13.605770 24150 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:24:13.606109 24137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fbe6a1abebc438bacde58b25c3a0b40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fbe6a1abebc438bacde58b25c3a0b40. Latest consensus state: current_term: 1 leader_uuid: "1fbe6a1abebc438bacde58b25c3a0b40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fbe6a1abebc438bacde58b25c3a0b40" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 40485 } } }
I20260926 06:24:13.606258 24137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fbe6a1abebc438bacde58b25c3a0b40 [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:13.606335 24150 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:24:13.606634 24133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:13.596147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48570 (local address 127.4.202.190:40485)
0926 06:24:13.596304 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:13.596312 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:13.596345 (+ 33us) server_negotiation.cc:407] Connection header received
0926 06:24:13.596774 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:13.596780 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:13.596822 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:13.596945 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:13.597721 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:13.598710 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:13.599748 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:13.600191 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:13.600313 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:13.602629 (+ 2316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:13.602632 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:13.602633 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:13.602643 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:13.604615 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:13.605017 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:13.605021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:13.605021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:13.605128 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:13.605336 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:13.605337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:13.605337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:13.605641 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:13.605796 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:13.606286 (+ 490us) server_negotiation.cc:299] Negotiation successful
0926 06:24:13.606414 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":74,"threads_started":1}
I20260926 06:24:13.607771 24150 catalog_manager.cc:1347] Generated new cluster ID: 20fdd88e85d5489e83659aab1df0b2bb
I20260926 06:24:13.607808 24150 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:24:13.628487 24150 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:24:13.628985 24150 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:24:13.639870 24150 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1fbe6a1abebc438bacde58b25c3a0b40: Generated new TSK 0
I20260926 06:24:13.640053 24150 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.202.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
Entry for principal kudu/127.4.202.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab.
I20260926 06:24:13.668990 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.129:0
--local_ip_for_outbound_sockets=127.4.202.129
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:40485
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:37305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:24:13.746369 24156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:13.746543 24156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:13.746565 24156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:13.747615 24156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:13.747648 24156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:13.747673 24156 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.129
I20260926 06:24:13.748971 24156 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:37305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.129.keytab
--principal=kudu/127.4.202.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:40485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24156
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.129
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:13.749128 24156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:13.749284 24156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:13.754262 24164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:13.754230 24162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:13.754199 24156 server_base.cc:1034] running on GCE node
W20260926 06:24:13.754216 24161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:13.754709 24156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:13.754962 24156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:13.756132 24156 hybrid_clock.cc:648] HybridClock initialized: now 1790403853756118 us; error 27 us; skew 500 ppm
Sep 26 06:24:13 dist-test-slave-pd0t krb5kdc[24049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403853, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:13.758522 24156 init.cc:377] Logged in from keytab as kudu/127.4.202.129@KRBTEST.COM (short username kudu)
I20260926 06:24:13.758945 24156 webserver.cc:460] Webserver started at http://127.4.202.129:35283/ using document root <none> and password file <none>
I20260926 06:24:13.759158 24156 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:13.759200 24156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:13.759334 24156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:13.760284 24156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-0/data/instance:
uuid: "439c41f0169548b0ba9e41ffd5be49eb"
format_stamp: "Formatted at 2026-09-26 06:24:13 on dist-test-slave-pd0t"
server_key: "89c567051295ff57c0fad88511551e3c"
server_key_iv: "2c6d42433bd39142b43cd48e68df8a1b"
server_key_version: "encryptionkey@0"
I20260926 06:24:13.760627 24156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance:
uuid: "439c41f0169548b0ba9e41ffd5be49eb"
format_stamp: "Formatted at 2026-09-26 06:24:13 on dist-test-slave-pd0t"
server_key: "89c567051295ff57c0fad88511551e3c"
server_key_iv: "2c6d42433bd39142b43cd48e68df8a1b"
server_key_version: "encryptionkey@0"
I20260926 06:24:13.761837 24156 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:24:13.762534 24171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:13.762727 24156 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:24:13.762812 24156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-0/wal
uuid: "439c41f0169548b0ba9e41ffd5be49eb"
format_stamp: "Formatted at 2026-09-26 06:24:13 on dist-test-slave-pd0t"
server_key: "89c567051295ff57c0fad88511551e3c"
server_key_iv: "2c6d42433bd39142b43cd48e68df8a1b"
server_key_version: "encryptionkey@0"
I20260926 06:24:13.762902 24156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:13.786031 24156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:13.787847 24156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:13.788012 24156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:13.788230 24156 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:24:13.788527 24156 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:24:13.788581 24156 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:13.788621 24156 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:24:13.788650 24156 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:13.794263 24156 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.129:37429
I20260926 06:24:13.794328 24284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.129:37429 every 8 connection(s)
I20260926 06:24:13.794682 24156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
Sep 26 06:24:13 dist-test-slave-pd0t krb5kdc[24049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403853, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.129@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:24:13.803321 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 24156
I20260926 06:24:13.803437 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance
I20260926 06:24:13.803670 4906 external_mini_cluster.cc:1428] Setting key a3ef4d2f38bfd57dead0f2af3b7f3416
I20260926 06:24:13.804004 24133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:13.795453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:51263 (local address 127.4.202.190:40485)
0926 06:24:13.795524 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:13.795527 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:13.795734 (+ 207us) server_negotiation.cc:407] Connection header received
0926 06:24:13.796281 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:13.796282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:13.796294 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:13.796348 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:13.797302 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:13.797828 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:13.798837 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:13.799066 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:13.799155 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:13.801226 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:13.801227 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:13.801227 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:13.801234 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:13.802453 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:13.802892 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:13.802892 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:13.802893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:13.802942 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:13.803239 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:13.803240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:13.803240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:13.803342 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:13.803424 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:13.803815 (+ 391us) server_negotiation.cc:299] Negotiation successful
0926 06:24:13.803909 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 06:24:13.804239 24287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:13.795425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:40485 (local address 127.4.202.129:51263)
0926 06:24:13.795600 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:13.795632 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:13.796166 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:13.796419 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:13.796427 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:13.796998 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:13.797180 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:13.797184 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:13.797928 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:13.797929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:13.798738 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:13.798740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:13.799172 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:13.799173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:13.799289 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:13.799805 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:24:13.799813 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:13.801140 (+ 1327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:13.802529 (+ 1389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:13.802535 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:13.802537 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:13.802828 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:13.803012 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:13.803013 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:13.803013 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:13.803172 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:13.803432 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:13.803439 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:13.803647 (+ 208us) client_negotiation.cc:769] Sending connection context
0926 06:24:13.803768 (+ 121us) client_negotiation.cc:240] Negotiation successful
0926 06:24:13.803950 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":97,"threads_started":1}
I20260926 06:24:13.804706 24285 heartbeater.cc:347] Connected to a master server at 127.4.202.190:40485
I20260926 06:24:13.804823 24285 heartbeater.cc:464] Registering TS with master...
I20260926 06:24:13.804994 24285 heartbeater.cc:511] Master 127.4.202.190:40485 requested a full tablet report, sending...
I20260926 06:24:13.805523 24094 ts_manager.cc:194] Registered new tserver with Master: 439c41f0169548b0ba9e41ffd5be49eb (127.4.202.129:37429)
I20260926 06:24:13.806142 24094 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.129@KRBTEST.COM'} at 127.4.202.129:51263
WARNING: no policy specified for kudu/127.4.202.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
Entry for principal kudu/127.4.202.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab.
I20260926 06:24:13.827587 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.130:0
--local_ip_for_outbound_sockets=127.4.202.130
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:40485
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:37305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:24:13.904779 24290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:13.905047 24290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:13.905092 24290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:13.906213 24290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:13.906277 24290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:13.906320 24290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.130
I20260926 06:24:13.907820 24290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:37305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.130.keytab
--principal=kudu/127.4.202.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:40485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.130
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:13.908077 24290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:13.908298 24290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:13.913721 24295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:13.913753 24296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:13.913753 24298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:13.913872 24290 server_base.cc:1034] running on GCE node
I20260926 06:24:13.914186 24290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:13.914424 24290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:13.915582 24290 hybrid_clock.cc:648] HybridClock initialized: now 1790403853915562 us; error 37 us; skew 500 ppm
Sep 26 06:24:13 dist-test-slave-pd0t krb5kdc[24049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403853, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:13.917901 24290 init.cc:377] Logged in from keytab as kudu/127.4.202.130@KRBTEST.COM (short username kudu)
I20260926 06:24:13.918265 24290 webserver.cc:460] Webserver started at http://127.4.202.130:35095/ using document root <none> and password file <none>
I20260926 06:24:13.918443 24290 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:13.918481 24290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:13.918565 24290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:13.919554 24290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-1/data/instance:
uuid: "e7bb49d43af6498f9a7e1203fc8a386d"
format_stamp: "Formatted at 2026-09-26 06:24:13 on dist-test-slave-pd0t"
server_key: "334ee9634e60166cbc8882e9a8634bfa"
server_key_iv: "03049fa3f49251149b2016d675ca4522"
server_key_version: "encryptionkey@0"
I20260926 06:24:13.919827 24290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance:
uuid: "e7bb49d43af6498f9a7e1203fc8a386d"
format_stamp: "Formatted at 2026-09-26 06:24:13 on dist-test-slave-pd0t"
server_key: "334ee9634e60166cbc8882e9a8634bfa"
server_key_iv: "03049fa3f49251149b2016d675ca4522"
server_key_version: "encryptionkey@0"
I20260926 06:24:13.920969 24290 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:24:13.921602 24305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:13.921743 24290 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:24:13.921823 24290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-1/wal
uuid: "e7bb49d43af6498f9a7e1203fc8a386d"
format_stamp: "Formatted at 2026-09-26 06:24:13 on dist-test-slave-pd0t"
server_key: "334ee9634e60166cbc8882e9a8634bfa"
server_key_iv: "03049fa3f49251149b2016d675ca4522"
server_key_version: "encryptionkey@0"
I20260926 06:24:13.921875 24290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:13.944416 24290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:13.946652 24290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:13.946753 24290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:13.946913 24290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:24:13.947209 24290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:24:13.947243 24290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:13.947263 24290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:24:13.947311 24290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:13.952749 24290 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.130:43527
I20260926 06:24:13.952809 24418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.130:43527 every 8 connection(s)
I20260926 06:24:13.953100 24290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
Sep 26 06:24:13 dist-test-slave-pd0t krb5kdc[24049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403853, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.130@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:24:13.961702 24133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:13.953788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:42493 (local address 127.4.202.190:40485)
0926 06:24:13.953867 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:13.953870 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:13.954037 (+ 167us) server_negotiation.cc:407] Connection header received
0926 06:24:13.954557 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:13.954558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:13.954570 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:13.954671 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:13.955567 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:13.956053 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:13.957018 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:13.957215 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:13.957298 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:13.959268 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:13.959269 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:13.959269 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:13.959276 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:13.960388 (+ 1112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:13.960773 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:13.960774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:13.960774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:13.960832 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:13.961085 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:13.961086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:13.961086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:13.961197 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:13.961271 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:13.961524 (+ 253us) server_negotiation.cc:299] Negotiation successful
0926 06:24:13.961617 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 06:24:13.961774 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 24290
I20260926 06:24:13.961865 24421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:13.953750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:40485 (local address 127.4.202.130:42493)
0926 06:24:13.953923 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:13.953955 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:13.954430 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:13.954746 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:13.954749 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:13.955286 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:13.955455 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:13.955459 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:13.956148 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:13.956149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:13.956917 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:13.956919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:13.957307 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:13.957308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:13.957429 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:13.957810 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:24:13.957818 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:13.959160 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:13.960481 (+ 1321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:13.960488 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:13.960490 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:13.960699 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:13.960900 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:13.960901 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:13.960901 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:13.961002 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:13.961263 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:13.961269 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:13.961397 (+ 128us) client_negotiation.cc:769] Sending connection context
0926 06:24:13.961502 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 06:24:13.961646 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":77,"threads_started":1}
I20260926 06:24:13.961884 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance
I20260926 06:24:13.962092 4906 external_mini_cluster.cc:1428] Setting key 1964c349644a3c4696a2a8c3824961d0
I20260926 06:24:13.962541 24419 heartbeater.cc:347] Connected to a master server at 127.4.202.190:40485
I20260926 06:24:13.962651 24419 heartbeater.cc:464] Registering TS with master...
I20260926 06:24:13.962852 24419 heartbeater.cc:511] Master 127.4.202.190:40485 requested a full tablet report, sending...
I20260926 06:24:13.963238 24094 ts_manager.cc:194] Registered new tserver with Master: e7bb49d43af6498f9a7e1203fc8a386d (127.4.202.130:43527)
I20260926 06:24:13.963619 24094 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.130@KRBTEST.COM'} at 127.4.202.130:42493
WARNING: no policy specified for kudu/127.4.202.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
Entry for principal kudu/127.4.202.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab.
I20260926 06:24:13.985873 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.131:0
--local_ip_for_outbound_sockets=127.4.202.131
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:40485
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:37305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:24:14.063548 24424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:14.063743 24424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:14.063764 24424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:14.064970 24424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:14.065011 24424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:14.065037 24424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.131
I20260926 06:24:14.066530 24424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:37305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.131.keytab
--principal=kudu/127.4.202.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:40485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24424
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.131
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:14.066780 24424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:14.066999 24424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:14.071846 24432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:14.071832 24429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:14.071946 24424 server_base.cc:1034] running on GCE node
W20260926 06:24:14.071833 24430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:14.072227 24424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:14.072456 24424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:14.073625 24424 hybrid_clock.cc:648] HybridClock initialized: now 1790403854073616 us; error 31 us; skew 500 ppm
Sep 26 06:24:14 dist-test-slave-pd0t krb5kdc[24049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403854, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:14.075958 24424 init.cc:377] Logged in from keytab as kudu/127.4.202.131@KRBTEST.COM (short username kudu)
I20260926 06:24:14.076402 24424 webserver.cc:460] Webserver started at http://127.4.202.131:39239/ using document root <none> and password file <none>
I20260926 06:24:14.076622 24424 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:14.076687 24424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:14.076843 24424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:14.077798 24424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-2/data/instance:
uuid: "b6a9010304f1449cb603e6db54fe49dc"
format_stamp: "Formatted at 2026-09-26 06:24:14 on dist-test-slave-pd0t"
server_key: "b8c0eb51725fbcf21da28033aa3b081a"
server_key_iv: "f79c44c66fb7896ff34b3c325e78c930"
server_key_version: "encryptionkey@0"
I20260926 06:24:14.078217 24424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance:
uuid: "b6a9010304f1449cb603e6db54fe49dc"
format_stamp: "Formatted at 2026-09-26 06:24:14 on dist-test-slave-pd0t"
server_key: "b8c0eb51725fbcf21da28033aa3b081a"
server_key_iv: "f79c44c66fb7896ff34b3c325e78c930"
server_key_version: "encryptionkey@0"
I20260926 06:24:14.079715 24424 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:24:14.080385 24439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:14.080551 24424 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:14.080649 24424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-2/wal
uuid: "b6a9010304f1449cb603e6db54fe49dc"
format_stamp: "Formatted at 2026-09-26 06:24:14 on dist-test-slave-pd0t"
server_key: "b8c0eb51725fbcf21da28033aa3b081a"
server_key_iv: "f79c44c66fb7896ff34b3c325e78c930"
server_key_version: "encryptionkey@0"
I20260926 06:24:14.080719 24424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:14.096634 24424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:14.098331 24424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:14.098493 24424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:14.098755 24424 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:24:14.099040 24424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:24:14.099092 24424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:14.099129 24424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:24:14.099157 24424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:14.105230 24424 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.131:37383
I20260926 06:24:14.105271 24552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.131:37383 every 8 connection(s)
I20260926 06:24:14.105571 24424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
I20260926 06:24:14.109997 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 24424
I20260926 06:24:14.110110 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance
I20260926 06:24:14.110302 4906 external_mini_cluster.cc:1428] Setting key 92eac17b587596d83788aa1980112230
Sep 26 06:24:14 dist-test-slave-pd0t krb5kdc[24049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403854, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.131@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:24:14.114976 24133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.106343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:53755 (local address 127.4.202.190:40485)
0926 06:24:14.106624 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:14.106627 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:14.106746 (+ 119us) server_negotiation.cc:407] Connection header received
0926 06:24:14.107241 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:14.107242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:14.107254 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:14.107314 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:14.108222 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.108719 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.109730 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.110001 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.110077 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:14.112404 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:14.112405 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:14.112406 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:14.112413 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:14.113554 (+ 1141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:14.113980 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:14.113981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:14.113982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:14.114041 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:14.114297 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:14.114298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:14.114298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:14.114410 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:14.114492 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:14.114809 (+ 317us) server_negotiation.cc:299] Negotiation successful
0926 06:24:14.114884 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240}
I20260926 06:24:14.115183 24555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.106237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:40485 (local address 127.4.202.131:53755)
0926 06:24:14.106581 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:14.106650 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:14.107136 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:14.107384 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:14.107387 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:14.107940 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:14.108121 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:14.108125 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.108815 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.108816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:14.109630 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:14.109632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.110109 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.110110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:14.110252 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:14.110842 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:24:14.110850 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:14.112315 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:14.113656 (+ 1341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:14.113663 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:14.113666 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:14.113880 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:14.114118 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:14.114119 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:14.114119 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:14.114222 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:14.114483 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:14.114489 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:14.114682 (+ 193us) client_negotiation.cc:769] Sending connection context
0926 06:24:14.114788 (+ 106us) client_negotiation.cc:240] Negotiation successful
0926 06:24:14.114972 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":91,"threads_started":1}
I20260926 06:24:14.115605 24553 heartbeater.cc:347] Connected to a master server at 127.4.202.190:40485
I20260926 06:24:14.115691 24553 heartbeater.cc:464] Registering TS with master...
I20260926 06:24:14.115823 24553 heartbeater.cc:511] Master 127.4.202.190:40485 requested a full tablet report, sending...
I20260926 06:24:14.116138 24094 ts_manager.cc:194] Registered new tserver with Master: b6a9010304f1449cb603e6db54fe49dc (127.4.202.131:37383)
I20260926 06:24:14.116513 24094 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.202.131@KRBTEST.COM'} at 127.4.202.131:53755
I20260926 06:24:14.121912 4906 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:24:14.128587 24133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.123221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48580 (local address 127.4.202.190:40485)
0926 06:24:14.123256 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:14.123257 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:14.123353 (+ 96us) server_negotiation.cc:407] Connection header received
0926 06:24:14.123391 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:14.123391 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:14.123399 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:14.123446 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:14.124017 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.124544 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.125223 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.125402 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.125470 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:14.126650 (+ 1180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:14.126651 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:14.126651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:14.126656 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:14.127738 (+ 1082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:14.128032 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:14.128032 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:14.128033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:14.128077 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:14.128226 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:14.128226 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:14.128227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:14.128340 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:14.128408 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:14.128486 (+ 78us) server_negotiation.cc:299] Negotiation successful
0926 06:24:14.128535 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 06:24:14.130153 24094 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48580:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:24:14.131744 24094 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:24:14.137889 24564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.133775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58106 (local address 127.4.202.130:43527)
0926 06:24:14.133969 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:14.133974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:14.134059 (+ 85us) server_negotiation.cc:407] Connection header received
0926 06:24:14.134111 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:14.134113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:14.134128 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:14.134212 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:24:14.134494 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.135657 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.136985 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.137454 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.137541 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:14.137600 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:14.137764 (+ 164us) server_negotiation.cc:299] Negotiation successful
0926 06:24:14.137805 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":75,"threads_started":1}
I20260926 06:24:14.137900 24565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.133831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:37429 (local address 127.0.0.1:53432)
0926 06:24:14.134263 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:14.134271 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:14.134340 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:14.134526 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:14.134527 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:14.134529 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:24:14.134600 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:14.134602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.135448 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.135449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:14.136678 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:14.136681 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.137666 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.137667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:14.137729 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:14.137735 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 06:24:14.137783 (+ 48us) client_negotiation.cc:240] Negotiation successful
0926 06:24:14.137825 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":91,"threads_started":1}
I20260926 06:24:14.137980 24563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.133691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52736 (local address 127.4.202.131:37383)
0926 06:24:14.134065 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:14.134068 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:14.134078 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:24:14.134110 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:14.134112 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:14.134126 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:14.134211 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:24:14.134441 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.135669 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.136811 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.137495 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.137566 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:14.137618 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:14.137863 (+ 245us) server_negotiation.cc:299] Negotiation successful
0926 06:24:14.137897 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":68,"threads_started":1}
I20260926 06:24:14.138132 24562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.133606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:37383 (local address 127.0.0.1:52736)
0926 06:24:14.133787 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:14.133824 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:14.133903 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:14.134289 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:14.134293 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:14.134295 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:24:14.134355 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:14.134357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.135790 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.135791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:14.136708 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:14.136710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.137668 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.137668 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:14.137729 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:14.137735 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 06:24:14.137783 (+ 48us) client_negotiation.cc:240] Negotiation successful
0926 06:24:14.137825 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":28,"thread_start_us":96,"threads_started":1}
I20260926 06:24:14.138149 24566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.134014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53432 (local address 127.4.202.129:37429)
0926 06:24:14.134218 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:14.134222 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:14.134331 (+ 109us) server_negotiation.cc:407] Connection header received
0926 06:24:14.134439 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:14.134442 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:14.134457 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:14.134531 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:24:14.134678 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.135344 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.136767 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.137495 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.137569 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:14.137619 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:14.138009 (+ 390us) server_negotiation.cc:299] Negotiation successful
0926 06:24:14.138061 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":77,"threads_started":1}
I20260926 06:24:14.138648 24561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.133646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:43527 (local address 127.0.0.1:58106)
0926 06:24:14.133972 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:14.133983 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:14.134046 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:14.134290 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:14.134293 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:14.134297 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:24:14.134413 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:14.134416 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.135786 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.135787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:14.136894 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:14.136895 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.137544 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.137545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:14.137625 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:14.137649 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 06:24:14.137747 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 06:24:14.137799 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":54,"thread_start_us":44,"threads_started":1}
I20260926 06:24:14.138830 24487 tablet_service.cc:1467] Processing CreateTablet for tablet e6e68053c34c436d91da7b408656c3d7 (DEFAULT_TABLE table=test-table [id=25de05e99f624404851058c4facd5443]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:24:14.138831 24219 tablet_service.cc:1467] Processing CreateTablet for tablet e6e68053c34c436d91da7b408656c3d7 (DEFAULT_TABLE table=test-table [id=25de05e99f624404851058c4facd5443]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:24:14.139148 24219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6e68053c34c436d91da7b408656c3d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:14.139148 24487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6e68053c34c436d91da7b408656c3d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:14.139283 24353 tablet_service.cc:1467] Processing CreateTablet for tablet e6e68053c34c436d91da7b408656c3d7 (DEFAULT_TABLE table=test-table [id=25de05e99f624404851058c4facd5443]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:24:14.139607 24353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6e68053c34c436d91da7b408656c3d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:14.142197 24567 tablet_bootstrap.cc:492] T e6e68053c34c436d91da7b408656c3d7 P e7bb49d43af6498f9a7e1203fc8a386d: Bootstrap starting.
I20260926 06:24:14.142611 24569 tablet_bootstrap.cc:492] T e6e68053c34c436d91da7b408656c3d7 P 439c41f0169548b0ba9e41ffd5be49eb: Bootstrap starting.
I20260926 06:24:14.142746 24568 tablet_bootstrap.cc:492] T e6e68053c34c436d91da7b408656c3d7 P b6a9010304f1449cb603e6db54fe49dc: Bootstrap starting.
I20260926 06:24:14.143067 24567 tablet_bootstrap.cc:654] T e6e68053c34c436d91da7b408656c3d7 P e7bb49d43af6498f9a7e1203fc8a386d: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:14.143244 24569 tablet_bootstrap.cc:654] T e6e68053c34c436d91da7b408656c3d7 P 439c41f0169548b0ba9e41ffd5be49eb: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:14.143363 24568 tablet_bootstrap.cc:654] T e6e68053c34c436d91da7b408656c3d7 P b6a9010304f1449cb603e6db54fe49dc: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:14.143684 24567 log.cc:824] T e6e68053c34c436d91da7b408656c3d7 P e7bb49d43af6498f9a7e1203fc8a386d: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:14.143819 24569 log.cc:824] T e6e68053c34c436d91da7b408656c3d7 P 439c41f0169548b0ba9e41ffd5be49eb: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:14.143857 24568 log.cc:824] T e6e68053c34c436d91da7b408656c3d7 P b6a9010304f1449cb603e6db54fe49dc: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:14.144452 24567 tablet_bootstrap.cc:492] T e6e68053c34c436d91da7b408656c3d7 P e7bb49d43af6498f9a7e1203fc8a386d: No bootstrap required, opened a new log
I20260926 06:24:14.144506 24567 ts_tablet_manager.cc:1397] T e6e68053c34c436d91da7b408656c3d7 P e7bb49d43af6498f9a7e1203fc8a386d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:24:14.144532 24568 tablet_bootstrap.cc:492] T e6e68053c34c436d91da7b408656c3d7 P b6a9010304f1449cb603e6db54fe49dc: No bootstrap required, opened a new log
I20260926 06:24:14.144603 24568 ts_tablet_manager.cc:1397] T e6e68053c34c436d91da7b408656c3d7 P b6a9010304f1449cb603e6db54fe49dc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:24:14.146071 24567 raft_consensus.cc:348] T e6e68053c34c436d91da7b408656c3d7 P e7bb49d43af6498f9a7e1203fc8a386d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6a9010304f1449cb603e6db54fe49dc" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37383 } } peers { permanent_uuid: "e7bb49d43af6498f9a7e1203fc8a386d" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43527 } } peers { permanent_uuid: "439c41f0169548b0ba9e41ffd5be49eb" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37429 } }
I20260926 06:24:14.146090 24568 raft_consensus.cc:348] T e6e68053c34c436d91da7b408656c3d7 P b6a9010304f1449cb603e6db54fe49dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6a9010304f1449cb603e6db54fe49dc" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37383 } } peers { permanent_uuid: "e7bb49d43af6498f9a7e1203fc8a386d" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43527 } } peers { permanent_uuid: "439c41f0169548b0ba9e41ffd5be49eb" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37429 } }
I20260926 06:24:14.146205 24567 raft_consensus.cc:374] T e6e68053c34c436d91da7b408656c3d7 P e7bb49d43af6498f9a7e1203fc8a386d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:14.146234 24568 raft_consensus.cc:374] T e6e68053c34c436d91da7b408656c3d7 P b6a9010304f1449cb603e6db54fe49dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:14.146234 24567 raft_consensus.cc:729] T e6e68053c34c436d91da7b408656c3d7 P e7bb49d43af6498f9a7e1203fc8a386d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7bb49d43af6498f9a7e1203fc8a386d, State: Initialized, Role: FOLLOWER
I20260926 06:24:14.146282 24568 raft_consensus.cc:729] T e6e68053c34c436d91da7b408656c3d7 P b6a9010304f1449cb603e6db54fe49dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6a9010304f1449cb603e6db54fe49dc, State: Initialized, Role: FOLLOWER
I20260926 06:24:14.146358 24567 consensus_queue.cc:260] T e6e68053c34c436d91da7b408656c3d7 P e7bb49d43af6498f9a7e1203fc8a386d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6a9010304f1449cb603e6db54fe49dc" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37383 } } peers { permanent_uuid: "e7bb49d43af6498f9a7e1203fc8a386d" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43527 } } peers { permanent_uuid: "439c41f0169548b0ba9e41ffd5be49eb" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37429 } }
I20260926 06:24:14.146389 24568 consensus_queue.cc:260] T e6e68053c34c436d91da7b408656c3d7 P b6a9010304f1449cb603e6db54fe49dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6a9010304f1449cb603e6db54fe49dc" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37383 } } peers { permanent_uuid: "e7bb49d43af6498f9a7e1203fc8a386d" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43527 } } peers { permanent_uuid: "439c41f0169548b0ba9e41ffd5be49eb" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37429 } }
I20260926 06:24:14.146685 24568 ts_tablet_manager.cc:1428] T e6e68053c34c436d91da7b408656c3d7 P b6a9010304f1449cb603e6db54fe49dc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:24:14.146685 24567 ts_tablet_manager.cc:1428] T e6e68053c34c436d91da7b408656c3d7 P e7bb49d43af6498f9a7e1203fc8a386d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:24:14.146757 24419 heartbeater.cc:503] Master 127.4.202.190:40485 was elected leader, sending a full tablet report...
I20260926 06:24:14.146818 24553 heartbeater.cc:503] Master 127.4.202.190:40485 was elected leader, sending a full tablet report...
I20260926 06:24:14.147387 24569 tablet_bootstrap.cc:492] T e6e68053c34c436d91da7b408656c3d7 P 439c41f0169548b0ba9e41ffd5be49eb: No bootstrap required, opened a new log
I20260926 06:24:14.147467 24569 ts_tablet_manager.cc:1397] T e6e68053c34c436d91da7b408656c3d7 P 439c41f0169548b0ba9e41ffd5be49eb: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260926 06:24:14.148839 24569 raft_consensus.cc:348] T e6e68053c34c436d91da7b408656c3d7 P 439c41f0169548b0ba9e41ffd5be49eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6a9010304f1449cb603e6db54fe49dc" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37383 } } peers { permanent_uuid: "e7bb49d43af6498f9a7e1203fc8a386d" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43527 } } peers { permanent_uuid: "439c41f0169548b0ba9e41ffd5be49eb" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37429 } }
I20260926 06:24:14.148978 24569 raft_consensus.cc:374] T e6e68053c34c436d91da7b408656c3d7 P 439c41f0169548b0ba9e41ffd5be49eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:14.149029 24569 raft_consensus.cc:729] T e6e68053c34c436d91da7b408656c3d7 P 439c41f0169548b0ba9e41ffd5be49eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 439c41f0169548b0ba9e41ffd5be49eb, State: Initialized, Role: FOLLOWER
I20260926 06:24:14.149137 24569 consensus_queue.cc:260] T e6e68053c34c436d91da7b408656c3d7 P 439c41f0169548b0ba9e41ffd5be49eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6a9010304f1449cb603e6db54fe49dc" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37383 } } peers { permanent_uuid: "e7bb49d43af6498f9a7e1203fc8a386d" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43527 } } peers { permanent_uuid: "439c41f0169548b0ba9e41ffd5be49eb" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37429 } }
I20260926 06:24:14.149406 24569 ts_tablet_manager.cc:1428] T e6e68053c34c436d91da7b408656c3d7 P 439c41f0169548b0ba9e41ffd5be49eb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:24:14.149451 24285 heartbeater.cc:503] Master 127.4.202.190:40485 was elected leader, sending a full tablet report...
W20260926 06:24:14.203622 24420 tablet.cc:2369] T e6e68053c34c436d91da7b408656c3d7 P e7bb49d43af6498f9a7e1203fc8a386d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 06:24:14.235841 24573 raft_consensus.cc:482] T e6e68053c34c436d91da7b408656c3d7 P b6a9010304f1449cb603e6db54fe49dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:24:14.235996 24573 raft_consensus.cc:504] T e6e68053c34c436d91da7b408656c3d7 P b6a9010304f1449cb603e6db54fe49dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6a9010304f1449cb603e6db54fe49dc" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37383 } } peers { permanent_uuid: "e7bb49d43af6498f9a7e1203fc8a386d" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43527 } } peers { permanent_uuid: "439c41f0169548b0ba9e41ffd5be49eb" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37429 } }
I20260926 06:24:14.236337 24573 leader_election.cc:288] T e6e68053c34c436d91da7b408656c3d7 P b6a9010304f1449cb603e6db54fe49dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7bb49d43af6498f9a7e1203fc8a386d (127.4.202.130:43527), 439c41f0169548b0ba9e41ffd5be49eb (127.4.202.129:37429)
I20260926 06:24:14.239275 24555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.236510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:43527 (local address 127.4.202.131:48587)
0926 06:24:14.236582 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:14.236595 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:14.236652 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:14.236805 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:14.236806 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:14.236808 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:24:14.236890 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:14.236892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.237642 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.237642 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:14.238437 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:14.238439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.239015 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.239016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:14.239092 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:14.239096 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:24:14.239157 (+ 61us) client_negotiation.cc:240] Negotiation successful
0926 06:24:14.239194 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":50}
I20260926 06:24:14.239281 24564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.236597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:48587 (local address 127.4.202.130:43527)
0926 06:24:14.236643 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:14.236645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:14.236655 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:24:14.236734 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:14.236734 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:14.236743 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:14.236795 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:24:14.236981 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.237554 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.238534 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.238929 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.238983 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:14.238990 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:14.239169 (+ 179us) server_negotiation.cc:299] Negotiation successful
0926 06:24:14.239194 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 06:24:14.239560 24566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.236698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:41167 (local address 127.4.202.129:37429)
0926 06:24:14.236741 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:14.236743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:14.236897 (+ 154us) server_negotiation.cc:407] Connection header received
0926 06:24:14.236962 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:14.236963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:14.236976 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:14.237025 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:24:14.237176 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.237642 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.238712 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.239188 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.239251 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:14.239258 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:14.239442 (+ 184us) server_negotiation.cc:299] Negotiation successful
0926 06:24:14.239494 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 06:24:14.239559 24373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e68053c34c436d91da7b408656c3d7" candidate_uuid: "b6a9010304f1449cb603e6db54fe49dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7bb49d43af6498f9a7e1203fc8a386d" is_pre_election: true
I20260926 06:24:14.239687 24373 raft_consensus.cc:2454] T e6e68053c34c436d91da7b408656c3d7 P e7bb49d43af6498f9a7e1203fc8a386d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6a9010304f1449cb603e6db54fe49dc in term 0.
I20260926 06:24:14.239760 24576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.236635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:37429 (local address 127.4.202.131:41167)
0926 06:24:14.236822 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:14.236831 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:14.236905 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:14.237048 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:14.237048 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:14.237050 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:24:14.237112 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:14.237113 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.237720 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.237721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:14.238458 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:14.238460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.239267 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.239268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:14.239340 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:14.239343 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:24:14.239643 (+ 300us) client_negotiation.cc:240] Negotiation successful
0926 06:24:14.239684 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":116,"threads_started":1}
I20260926 06:24:14.239945 24443 leader_election.cc:302] T e6e68053c34c436d91da7b408656c3d7 P b6a9010304f1449cb603e6db54fe49dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b6a9010304f1449cb603e6db54fe49dc, e7bb49d43af6498f9a7e1203fc8a386d; no voters:
I20260926 06:24:14.240072 24573 raft_consensus.cc:2781] T e6e68053c34c436d91da7b408656c3d7 P b6a9010304f1449cb603e6db54fe49dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:24:14.240160 24573 raft_consensus.cc:482] T e6e68053c34c436d91da7b408656c3d7 P b6a9010304f1449cb603e6db54fe49dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:24:14.240203 24573 raft_consensus.cc:3037] T e6e68053c34c436d91da7b408656c3d7 P b6a9010304f1449cb603e6db54fe49dc [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:14.240279 24239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e68053c34c436d91da7b408656c3d7" candidate_uuid: "b6a9010304f1449cb603e6db54fe49dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "439c41f0169548b0ba9e41ffd5be49eb" is_pre_election: true
I20260926 06:24:14.240406 24239 raft_consensus.cc:2454] T e6e68053c34c436d91da7b408656c3d7 P 439c41f0169548b0ba9e41ffd5be49eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6a9010304f1449cb603e6db54fe49dc in term 0.
I20260926 06:24:14.241031 24573 raft_consensus.cc:504] T e6e68053c34c436d91da7b408656c3d7 P b6a9010304f1449cb603e6db54fe49dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6a9010304f1449cb603e6db54fe49dc" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37383 } } peers { permanent_uuid: "e7bb49d43af6498f9a7e1203fc8a386d" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43527 } } peers { permanent_uuid: "439c41f0169548b0ba9e41ffd5be49eb" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37429 } }
I20260926 06:24:14.241166 24573 leader_election.cc:288] T e6e68053c34c436d91da7b408656c3d7 P b6a9010304f1449cb603e6db54fe49dc [CANDIDATE]: Term 1 election: Requested vote from peers e7bb49d43af6498f9a7e1203fc8a386d (127.4.202.130:43527), 439c41f0169548b0ba9e41ffd5be49eb (127.4.202.129:37429)
I20260926 06:24:14.241318 24373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e68053c34c436d91da7b408656c3d7" candidate_uuid: "b6a9010304f1449cb603e6db54fe49dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7bb49d43af6498f9a7e1203fc8a386d"
I20260926 06:24:14.241341 24239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e68053c34c436d91da7b408656c3d7" candidate_uuid: "b6a9010304f1449cb603e6db54fe49dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "439c41f0169548b0ba9e41ffd5be49eb"
I20260926 06:24:14.241412 24239 raft_consensus.cc:3037] T e6e68053c34c436d91da7b408656c3d7 P 439c41f0169548b0ba9e41ffd5be49eb [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:14.241412 24373 raft_consensus.cc:3037] T e6e68053c34c436d91da7b408656c3d7 P e7bb49d43af6498f9a7e1203fc8a386d [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:14.242265 24239 raft_consensus.cc:2454] T e6e68053c34c436d91da7b408656c3d7 P 439c41f0169548b0ba9e41ffd5be49eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6a9010304f1449cb603e6db54fe49dc in term 1.
I20260926 06:24:14.242265 24373 raft_consensus.cc:2454] T e6e68053c34c436d91da7b408656c3d7 P e7bb49d43af6498f9a7e1203fc8a386d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6a9010304f1449cb603e6db54fe49dc in term 1.
I20260926 06:24:14.242430 24443 leader_election.cc:302] T e6e68053c34c436d91da7b408656c3d7 P b6a9010304f1449cb603e6db54fe49dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b6a9010304f1449cb603e6db54fe49dc, e7bb49d43af6498f9a7e1203fc8a386d; no voters:
I20260926 06:24:14.242532 24573 raft_consensus.cc:2781] T e6e68053c34c436d91da7b408656c3d7 P b6a9010304f1449cb603e6db54fe49dc [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:24:14.242697 24573 raft_consensus.cc:686] T e6e68053c34c436d91da7b408656c3d7 P b6a9010304f1449cb603e6db54fe49dc [term 1 LEADER]: Becoming Leader. State: Replica: b6a9010304f1449cb603e6db54fe49dc, State: Running, Role: LEADER
I20260926 06:24:14.242786 24573 consensus_queue.cc:237] T e6e68053c34c436d91da7b408656c3d7 P b6a9010304f1449cb603e6db54fe49dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6a9010304f1449cb603e6db54fe49dc" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37383 } } peers { permanent_uuid: "e7bb49d43af6498f9a7e1203fc8a386d" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43527 } } peers { permanent_uuid: "439c41f0169548b0ba9e41ffd5be49eb" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37429 } }
I20260926 06:24:14.243537 24093 catalog_manager.cc:5495] T e6e68053c34c436d91da7b408656c3d7 P b6a9010304f1449cb603e6db54fe49dc reported cstate change: term changed from 0 to 1, leader changed from <none> to b6a9010304f1449cb603e6db54fe49dc (127.4.202.131). New cstate: current_term: 1 leader_uuid: "b6a9010304f1449cb603e6db54fe49dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6a9010304f1449cb603e6db54fe49dc" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 37383 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e7bb49d43af6498f9a7e1203fc8a386d" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "439c41f0169548b0ba9e41ffd5be49eb" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37429 } health_report { overall_health: UNKNOWN } } }
I20260926 06:24:14.251348 24563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.249020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52742 (local address 127.4.202.131:37383)
0926 06:24:14.249084 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:14.249085 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:14.249094 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:24:14.249144 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:14.249145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:14.249161 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:14.249224 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:24:14.249351 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.249786 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.250553 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.250772 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.250847 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:14.251019 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:24:14.251121 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:24:14.251214 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:14.251239 (+ 25us) server_negotiation.cc:299] Negotiation successful
0926 06:24:14.251257 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 06:24:14.252971 24373 raft_consensus.cc:1261] T e6e68053c34c436d91da7b408656c3d7 P e7bb49d43af6498f9a7e1203fc8a386d [term 1 FOLLOWER]: Refusing update from remote peer b6a9010304f1449cb603e6db54fe49dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:24:14.252971 24239 raft_consensus.cc:1261] T e6e68053c34c436d91da7b408656c3d7 P 439c41f0169548b0ba9e41ffd5be49eb [term 1 FOLLOWER]: Refusing update from remote peer b6a9010304f1449cb603e6db54fe49dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:24:14.253294 24573 consensus_queue.cc:1035] T e6e68053c34c436d91da7b408656c3d7 P b6a9010304f1449cb603e6db54fe49dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7bb49d43af6498f9a7e1203fc8a386d" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 43527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:24:14.253439 24577 consensus_queue.cc:1035] T e6e68053c34c436d91da7b408656c3d7 P b6a9010304f1449cb603e6db54fe49dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "439c41f0169548b0ba9e41ffd5be49eb" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:24:14.255645 24581 mvcc.cc:205] Tried to move back new op lower bound from 7333494187017637888 to 7333494186979082240. Current Snapshot: MvccSnapshot[applied={T|T < 7333494187017637888}]
I20260926 06:24:14.256392 24582 mvcc.cc:205] Tried to move back new op lower bound from 7333494187017637888 to 7333494186979082240. Current Snapshot: MvccSnapshot[applied={T|T < 7333494187017637888 or (T in {7333494187017637888})}]
I20260926 06:24:14.257701 24093 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:24:14.257776 24093 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:24:14.258536 24093 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1fbe6a1abebc438bacde58b25c3a0b40: Sending DeleteTablet for 3 replicas of tablet e6e68053c34c436d91da7b408656c3d7
I20260926 06:24:14.258864 24487 tablet_service.cc:1514] Processing DeleteTablet for tablet e6e68053c34c436d91da7b408656c3d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:24:14 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:52736
I20260926 06:24:14.258908 24353 tablet_service.cc:1514] Processing DeleteTablet for tablet e6e68053c34c436d91da7b408656c3d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:24:14 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:58106
I20260926 06:24:14.258972 24219 tablet_service.cc:1514] Processing DeleteTablet for tablet e6e68053c34c436d91da7b408656c3d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:24:14 UTC) from {username='kudu', principal='kudu/127.4.202.190@KRBTEST.COM'} at 127.0.0.1:53432
I20260926 06:24:14.259145 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 24156
I20260926 06:24:14.259195 24591 tablet_replica.cc:330] T e6e68053c34c436d91da7b408656c3d7 P e7bb49d43af6498f9a7e1203fc8a386d: stopping tablet replica
I20260926 06:24:14.259285 24591 raft_consensus.cc:2229] T e6e68053c34c436d91da7b408656c3d7 P e7bb49d43af6498f9a7e1203fc8a386d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:24:14.259382 24591 raft_consensus.cc:2258] T e6e68053c34c436d91da7b408656c3d7 P e7bb49d43af6498f9a7e1203fc8a386d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:24:14.259832 24591 ts_tablet_manager.cc:1905] T e6e68053c34c436d91da7b408656c3d7 P e7bb49d43af6498f9a7e1203fc8a386d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:24:14.261719 24591 ts_tablet_manager.cc:1918] T e6e68053c34c436d91da7b408656c3d7 P e7bb49d43af6498f9a7e1203fc8a386d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:24:14.261793 24591 log.cc:1196] T e6e68053c34c436d91da7b408656c3d7 P e7bb49d43af6498f9a7e1203fc8a386d: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-1/wal/wals/e6e68053c34c436d91da7b408656c3d7
I20260926 06:24:14.262058 24591 ts_tablet_manager.cc:1939] T e6e68053c34c436d91da7b408656c3d7 P e7bb49d43af6498f9a7e1203fc8a386d: Deleting consensus metadata
I20260926 06:24:14.262646 24081 catalog_manager.cc:4841] TS e7bb49d43af6498f9a7e1203fc8a386d (127.4.202.130:43527): tablet e6e68053c34c436d91da7b408656c3d7 (table test-table [id=25de05e99f624404851058c4facd5443]) successfully deleted
I20260926 06:24:14.263089 24590 tablet_replica.cc:330] T e6e68053c34c436d91da7b408656c3d7 P b6a9010304f1449cb603e6db54fe49dc: stopping tablet replica
I20260926 06:24:14.263163 24590 raft_consensus.cc:2229] T e6e68053c34c436d91da7b408656c3d7 P b6a9010304f1449cb603e6db54fe49dc [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:24:14.263268 24590 raft_consensus.cc:2258] T e6e68053c34c436d91da7b408656c3d7 P b6a9010304f1449cb603e6db54fe49dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:24:14.263665 24590 ts_tablet_manager.cc:1905] T e6e68053c34c436d91da7b408656c3d7 P b6a9010304f1449cb603e6db54fe49dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:24:14.265573 24590 ts_tablet_manager.cc:1918] T e6e68053c34c436d91da7b408656c3d7 P b6a9010304f1449cb603e6db54fe49dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:24:14.265646 24590 log.cc:1196] T e6e68053c34c436d91da7b408656c3d7 P b6a9010304f1449cb603e6db54fe49dc: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790403591640726-4906-0/minicluster-data/ts-2/wal/wals/e6e68053c34c436d91da7b408656c3d7
I20260926 06:24:14.265981 24590 ts_tablet_manager.cc:1939] T e6e68053c34c436d91da7b408656c3d7 P b6a9010304f1449cb603e6db54fe49dc: Deleting consensus metadata
I20260926 06:24:14.266450 24078 catalog_manager.cc:4841] TS b6a9010304f1449cb603e6db54fe49dc (127.4.202.131:37383): tablet e6e68053c34c436d91da7b408656c3d7 (table test-table [id=25de05e99f624404851058c4facd5443]) successfully deleted
W20260926 06:24:14.267340 24081 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.202.129:37429 (error 108)
I20260926 06:24:14.267608 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 24290
I20260926 06:24:14.268131 24561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.267659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:37429 (local address 127.0.0.1:53440)
0926 06:24:14.268061 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:14.268101 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.129:37429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380}
W20260926 06:24:14.268257 24081 catalog_manager.cc:4584] TS 439c41f0169548b0ba9e41ffd5be49eb (127.4.202.129:37429): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e6e68053c34c436d91da7b408656c3d7: Network error: Client connection negotiation failed: client connection to 127.4.202.129:37429: connect: Connection refused (error 111)
I20260926 06:24:14.273327 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 24424
I20260926 06:24:14.278017 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 24062
2026-09-26T06:24:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1919 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-26T06:24:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:24:14Z Disabled control of system clock
I20260926 06:24:14.304729 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:34333
--webserver_interface=127.4.202.190
--webserver_port=0
--builtin_ntp_servers=127.4.202.148:40813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:34333
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 06:24:14.386188 24597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:14.386384 24597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:14.386404 24597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:14.387548 24597 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:24:14.387591 24597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:14.387604 24597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:14.387616 24597 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:24:14.387627 24597 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:24:14.389001 24597 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:40813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:34333
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:34333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:14.389245 24597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:14.389528 24597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:14.394232 24602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:14.394222 24603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:14.394233 24605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:14.394548 24597 server_base.cc:1034] running on GCE node
I20260926 06:24:14.394793 24597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:14.395018 24597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:14.396152 24597 hybrid_clock.cc:648] HybridClock initialized: now 1790403854396139 us; error 28 us; skew 500 ppm
I20260926 06:24:14.397359 24597 webserver.cc:460] Webserver started at http://127.4.202.190:35185/ using document root <none> and password file <none>
I20260926 06:24:14.397604 24597 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:14.397648 24597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:14.397789 24597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:14.398901 24597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "7107ae4085ee4140b09af771a576817e"
format_stamp: "Formatted at 2026-09-26 06:24:14 on dist-test-slave-pd0t"
server_key: "ad868d00cb9577c20b47402ad73e43c9"
server_key_iv: "10fab2ae79460be81d71bcf29ed515ca"
server_key_version: "encryptionkey@0"
I20260926 06:24:14.399248 24597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "7107ae4085ee4140b09af771a576817e"
format_stamp: "Formatted at 2026-09-26 06:24:14 on dist-test-slave-pd0t"
server_key: "ad868d00cb9577c20b47402ad73e43c9"
server_key_iv: "10fab2ae79460be81d71bcf29ed515ca"
server_key_version: "encryptionkey@0"
I20260926 06:24:14.400527 24597 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:24:14.401216 24611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:14.401414 24597 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:24:14.401486 24597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "7107ae4085ee4140b09af771a576817e"
format_stamp: "Formatted at 2026-09-26 06:24:14 on dist-test-slave-pd0t"
server_key: "ad868d00cb9577c20b47402ad73e43c9"
server_key_iv: "10fab2ae79460be81d71bcf29ed515ca"
server_key_version: "encryptionkey@0"
I20260926 06:24:14.401578 24597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:14.427211 24597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:14.427480 24597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:14.427585 24597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:14.431191 24597 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:34333
I20260926 06:24:14.431243 24663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:34333 every 8 connection(s)
I20260926 06:24:14.431553 24597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:24:14.432526 24664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:14.434803 24664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7107ae4085ee4140b09af771a576817e: Bootstrap starting.
I20260926 06:24:14.435364 24664 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7107ae4085ee4140b09af771a576817e: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:14.435658 24664 log.cc:824] T 00000000000000000000000000000000 P 7107ae4085ee4140b09af771a576817e: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:14.436313 24664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7107ae4085ee4140b09af771a576817e: No bootstrap required, opened a new log
I20260926 06:24:14.437825 24664 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7107ae4085ee4140b09af771a576817e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7107ae4085ee4140b09af771a576817e" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34333 } }
I20260926 06:24:14.438004 24664 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7107ae4085ee4140b09af771a576817e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:14.438056 24664 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7107ae4085ee4140b09af771a576817e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7107ae4085ee4140b09af771a576817e, State: Initialized, Role: FOLLOWER
I20260926 06:24:14.438237 24664 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7107ae4085ee4140b09af771a576817e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7107ae4085ee4140b09af771a576817e" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34333 } }
I20260926 06:24:14.438335 24664 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7107ae4085ee4140b09af771a576817e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:24:14.438380 24664 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7107ae4085ee4140b09af771a576817e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:24:14.438427 24664 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7107ae4085ee4140b09af771a576817e [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:14.438849 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 24597
I20260926 06:24:14.438967 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:24:14.439188 4906 external_mini_cluster.cc:1428] Setting key 87aca72ae1bf5de8216d6a00fd1469e3
I20260926 06:24:14.439164 24664 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7107ae4085ee4140b09af771a576817e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7107ae4085ee4140b09af771a576817e" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34333 } }
I20260926 06:24:14.439270 24664 leader_election.cc:302] T 00000000000000000000000000000000 P 7107ae4085ee4140b09af771a576817e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7107ae4085ee4140b09af771a576817e; no voters:
I20260926 06:24:14.439433 24664 leader_election.cc:288] T 00000000000000000000000000000000 P 7107ae4085ee4140b09af771a576817e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:24:14.439472 24667 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7107ae4085ee4140b09af771a576817e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:24:14.439601 24667 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7107ae4085ee4140b09af771a576817e [term 1 LEADER]: Becoming Leader. State: Replica: 7107ae4085ee4140b09af771a576817e, State: Running, Role: LEADER
I20260926 06:24:14.439688 24664 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7107ae4085ee4140b09af771a576817e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:24:14.439693 24667 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7107ae4085ee4140b09af771a576817e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7107ae4085ee4140b09af771a576817e" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34333 } }
I20260926 06:24:14.440022 24667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7107ae4085ee4140b09af771a576817e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7107ae4085ee4140b09af771a576817e. Latest consensus state: current_term: 1 leader_uuid: "7107ae4085ee4140b09af771a576817e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7107ae4085ee4140b09af771a576817e" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34333 } } }
I20260926 06:24:14.440099 24667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7107ae4085ee4140b09af771a576817e [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:14.440126 24669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7107ae4085ee4140b09af771a576817e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7107ae4085ee4140b09af771a576817e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7107ae4085ee4140b09af771a576817e" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34333 } } }
I20260926 06:24:14.440212 24669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7107ae4085ee4140b09af771a576817e [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:14.440366 24675 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:24:14.440908 24675 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:24:14.442503 24675 catalog_manager.cc:1347] Generated new cluster ID: 9b3a7499f9f5449483e6579f2e136804
I20260926 06:24:14.442559 24675 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:24:14.445497 24671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.439708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48864 (local address 127.4.202.190:34333)
0926 06:24:14.440374 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:14.440383 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:14.440414 (+ 31us) server_negotiation.cc:407] Connection header received
0926 06:24:14.440525 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:14.440532 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:14.440600 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:14.440993 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:14.441103 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.442660 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.443435 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.444038 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.444148 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:14.444844 (+ 696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:14.444848 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:14.444848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:14.445026 (+ 178us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:14.445058 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:14.445069 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:14.445139 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:14.445213 (+ 74us) server_negotiation.cc:299] Negotiation successful
0926 06:24:14.445292 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":80,"threads_started":1}
I20260926 06:24:14.455127 24675 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:24:14.455776 24675 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:24:14.460326 24675 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7107ae4085ee4140b09af771a576817e: Generated new TSK 0
I20260926 06:24:14.460484 24675 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:24:14.465360 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.129:0
--local_ip_for_outbound_sockets=127.4.202.129
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:34333
--builtin_ntp_servers=127.4.202.148:40813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 06:24:14.548193 24688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:14.548411 24688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:14.548447 24688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:14.549558 24688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:14.549619 24688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:14.549661 24688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.129
I20260926 06:24:14.552127 24688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:40813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.202.129
--webserver_port=0
--tserver_master_addrs=127.4.202.190:34333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.129
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:14.552389 24688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:14.552747 24688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:14.558993 24693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:14.559137 24694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:14.559190 24696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:14.559165 24688 server_base.cc:1034] running on GCE node
I20260926 06:24:14.559463 24688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:14.559676 24688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:14.560815 24688 hybrid_clock.cc:648] HybridClock initialized: now 1790403854560795 us; error 37 us; skew 500 ppm
I20260926 06:24:14.562068 24688 webserver.cc:460] Webserver started at http://127.4.202.129:44909/ using document root <none> and password file <none>
I20260926 06:24:14.562323 24688 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:14.562391 24688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:14.562551 24688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:14.563644 24688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-0/data/instance:
uuid: "3280b77304e84a94b745365753cf400e"
format_stamp: "Formatted at 2026-09-26 06:24:14 on dist-test-slave-pd0t"
server_key: "4746a11a800fd49cd8c920b741f6eedb"
server_key_iv: "9b208e1573d3663d2a22ac643ec5f9da"
server_key_version: "encryptionkey@0"
I20260926 06:24:14.563997 24688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance:
uuid: "3280b77304e84a94b745365753cf400e"
format_stamp: "Formatted at 2026-09-26 06:24:14 on dist-test-slave-pd0t"
server_key: "4746a11a800fd49cd8c920b741f6eedb"
server_key_iv: "9b208e1573d3663d2a22ac643ec5f9da"
server_key_version: "encryptionkey@0"
I20260926 06:24:14.565282 24688 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 06:24:14.566039 24702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:14.566236 24688 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 06:24:14.566339 24688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-0/wal
uuid: "3280b77304e84a94b745365753cf400e"
format_stamp: "Formatted at 2026-09-26 06:24:14 on dist-test-slave-pd0t"
server_key: "4746a11a800fd49cd8c920b741f6eedb"
server_key_iv: "9b208e1573d3663d2a22ac643ec5f9da"
server_key_version: "encryptionkey@0"
I20260926 06:24:14.566431 24688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:14.575861 24688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:14.576133 24688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:14.576267 24688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:14.576478 24688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:24:14.576776 24688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:24:14.576826 24688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:14.576866 24688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:24:14.576892 24688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:14.582232 24688 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.129:35389
I20260926 06:24:14.582290 24815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.129:35389 every 8 connection(s)
I20260926 06:24:14.582667 24688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-0/data/info.pb
I20260926 06:24:14.583259 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 24688
I20260926 06:24:14.583367 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-0/wal/instance
I20260926 06:24:14.583587 4906 external_mini_cluster.cc:1428] Setting key 6d6c8b30aa25feb6f2e30a9d6bdcc4f1
I20260926 06:24:14.584474 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.130:0
--local_ip_for_outbound_sockets=127.4.202.130
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:34333
--builtin_ntp_servers=127.4.202.148:40813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 06:24:14.588090 24671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.583430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.129:48055 (local address 127.4.202.190:34333)
0926 06:24:14.583491 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:14.583493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:14.583738 (+ 245us) server_negotiation.cc:407] Connection header received
0926 06:24:14.584360 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:14.584361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:14.584374 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:14.584430 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:14.585172 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.585666 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.586679 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.586891 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.586979 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:14.587830 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:14.587832 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:14.587832 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:14.587836 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:14.587850 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:14.587854 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:14.587903 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:14.587986 (+ 83us) server_negotiation.cc:299] Negotiation successful
0926 06:24:14.588014 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 06:24:14.588295 24818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.583425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34333 (local address 127.4.202.129:48055)
0926 06:24:14.583607 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:14.583662 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:14.584263 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:14.584516 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:14.584520 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:14.584530 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:14.585070 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:14.585075 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.585775 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.585776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:14.586560 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:14.586563 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.586995 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.586996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:14.587133 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:14.587679 (+ 546us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:24:14.587680 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:14.587706 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:24:14.587727 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:14.587910 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:14.587915 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:14.587918 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:24:14.587978 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 06:24:14.588073 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":83,"threads_started":1}
I20260926 06:24:14.588799 24816 heartbeater.cc:347] Connected to a master server at 127.4.202.190:34333
I20260926 06:24:14.588909 24816 heartbeater.cc:464] Registering TS with master...
I20260926 06:24:14.589085 24816 heartbeater.cc:511] Master 127.4.202.190:34333 requested a full tablet report, sending...
I20260926 06:24:14.589608 24628 ts_manager.cc:194] Registered new tserver with Master: 3280b77304e84a94b745365753cf400e (127.4.202.129:35389)
I20260926 06:24:14.590219 24628 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.202.129:48055
W20260926 06:24:14.663441 24819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:14.663713 24819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:14.663750 24819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:14.664892 24819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:14.664955 24819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:14.664999 24819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.130
I20260926 06:24:14.666339 24819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:40813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.202.130
--webserver_port=0
--tserver_master_addrs=127.4.202.190:34333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.130
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:14.666633 24819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:14.666920 24819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:14.671800 24825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:14.671803 24824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:14.671890 24827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:14.672086 24819 server_base.cc:1034] running on GCE node
I20260926 06:24:14.672290 24819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:14.672477 24819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:14.673629 24819 hybrid_clock.cc:648] HybridClock initialized: now 1790403854673617 us; error 26 us; skew 500 ppm
I20260926 06:24:14.675021 24819 webserver.cc:460] Webserver started at http://127.4.202.130:43957/ using document root <none> and password file <none>
I20260926 06:24:14.675268 24819 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:14.675315 24819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:14.675402 24819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:14.676337 24819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-1/data/instance:
uuid: "9cd7ce62127e4b708d1721583f3b350e"
format_stamp: "Formatted at 2026-09-26 06:24:14 on dist-test-slave-pd0t"
server_key: "182af620bf1a6c013e923407ffd23450"
server_key_iv: "4fa26527c2245bc38a4d3c0e39154919"
server_key_version: "encryptionkey@0"
I20260926 06:24:14.676613 24819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance:
uuid: "9cd7ce62127e4b708d1721583f3b350e"
format_stamp: "Formatted at 2026-09-26 06:24:14 on dist-test-slave-pd0t"
server_key: "182af620bf1a6c013e923407ffd23450"
server_key_iv: "4fa26527c2245bc38a4d3c0e39154919"
server_key_version: "encryptionkey@0"
I20260926 06:24:14.677778 24819 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:24:14.678496 24833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:14.678705 24819 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:24:14.678769 24819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-1/wal
uuid: "9cd7ce62127e4b708d1721583f3b350e"
format_stamp: "Formatted at 2026-09-26 06:24:14 on dist-test-slave-pd0t"
server_key: "182af620bf1a6c013e923407ffd23450"
server_key_iv: "4fa26527c2245bc38a4d3c0e39154919"
server_key_version: "encryptionkey@0"
I20260926 06:24:14.678850 24819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:14.688506 24819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:14.688787 24819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:14.688920 24819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:14.689127 24819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:24:14.689417 24819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:24:14.689450 24819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:14.689471 24819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:24:14.689515 24819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:14.695026 24819 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.130:39879
I20260926 06:24:14.695096 24946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.130:39879 every 8 connection(s)
I20260926 06:24:14.695380 24819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-1/data/info.pb
I20260926 06:24:14.698803 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 24819
I20260926 06:24:14.698905 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-1/wal/instance
I20260926 06:24:14.699129 4906 external_mini_cluster.cc:1428] Setting key 3200dc0a9530462b14b81e2dd5f81e7a
I20260926 06:24:14.700141 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.202.131:0
--local_ip_for_outbound_sockets=127.4.202.131
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:34333
--builtin_ntp_servers=127.4.202.148:40813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 06:24:14.700691 24671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.696098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.130:44719 (local address 127.4.202.190:34333)
0926 06:24:14.696162 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:14.696165 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:14.696369 (+ 204us) server_negotiation.cc:407] Connection header received
0926 06:24:14.696796 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:14.696798 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:14.696808 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:14.696859 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:14.697445 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.697967 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.699161 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.699483 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.699552 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:14.700430 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:14.700431 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:14.700432 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:14.700438 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:14.700460 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:14.700464 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:14.700511 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:14.700615 (+ 104us) server_negotiation.cc:299] Negotiation successful
0926 06:24:14.700636 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 06:24:14.700966 24949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.696083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34333 (local address 127.4.202.130:44719)
0926 06:24:14.696262 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:14.696292 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:14.696719 (+ 427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:14.696903 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:14.696906 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:14.696914 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:14.697378 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:14.697381 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.698044 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.698045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:14.699074 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:14.699077 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.699564 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.699565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:14.699701 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:14.700248 (+ 547us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:24:14.700248 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:14.700287 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:24:14.700314 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:14.700531 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:14.700536 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:14.700539 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:24:14.700611 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 06:24:14.700720 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":85,"threads_started":1}
I20260926 06:24:14.701856 24947 heartbeater.cc:347] Connected to a master server at 127.4.202.190:34333
I20260926 06:24:14.701954 24947 heartbeater.cc:464] Registering TS with master...
I20260926 06:24:14.702116 24947 heartbeater.cc:511] Master 127.4.202.190:34333 requested a full tablet report, sending...
I20260926 06:24:14.702517 24628 ts_manager.cc:194] Registered new tserver with Master: 9cd7ce62127e4b708d1721583f3b350e (127.4.202.130:39879)
I20260926 06:24:14.703068 24628 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.202.130:44719
W20260926 06:24:14.778355 24950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:14.778538 24950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:14.778558 24950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:14.779663 24950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:14.779702 24950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:14.779726 24950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.202.131
I20260926 06:24:14.780925 24950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:40813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.202.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.202.131
--webserver_port=0
--tserver_master_addrs=127.4.202.190:34333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.202.131
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:14.781152 24950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:14.781384 24950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:14.786105 24956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:14.786132 24958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:14.786358 24950 server_base.cc:1034] running on GCE node
W20260926 06:24:14.786450 24955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:14.786712 24950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:14.786958 24950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:14.788101 24950 hybrid_clock.cc:648] HybridClock initialized: now 1790403854788082 us; error 37 us; skew 500 ppm
I20260926 06:24:14.789356 24950 webserver.cc:460] Webserver started at http://127.4.202.131:35245/ using document root <none> and password file <none>
I20260926 06:24:14.789606 24950 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:14.789652 24950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:14.789790 24950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:14.790935 24950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-2/data/instance:
uuid: "5dc2fe3dc4c745cf932d91b5274a37f3"
format_stamp: "Formatted at 2026-09-26 06:24:14 on dist-test-slave-pd0t"
server_key: "ff8b5816d29cd4e6f27fd70a704d2bd1"
server_key_iv: "d947bd3f359e65c7555de75f8ce69003"
server_key_version: "encryptionkey@0"
I20260926 06:24:14.791301 24950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance:
uuid: "5dc2fe3dc4c745cf932d91b5274a37f3"
format_stamp: "Formatted at 2026-09-26 06:24:14 on dist-test-slave-pd0t"
server_key: "ff8b5816d29cd4e6f27fd70a704d2bd1"
server_key_iv: "d947bd3f359e65c7555de75f8ce69003"
server_key_version: "encryptionkey@0"
I20260926 06:24:14.792531 24950 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:24:14.793207 24964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:14.793375 24950 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:24:14.793474 24950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-2/wal
uuid: "5dc2fe3dc4c745cf932d91b5274a37f3"
format_stamp: "Formatted at 2026-09-26 06:24:14 on dist-test-slave-pd0t"
server_key: "ff8b5816d29cd4e6f27fd70a704d2bd1"
server_key_iv: "d947bd3f359e65c7555de75f8ce69003"
server_key_version: "encryptionkey@0"
I20260926 06:24:14.793552 24950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:14.803853 24950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:14.804100 24950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:14.804272 24950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:14.804484 24950 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 06:24:14.804824 24950 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 06:24:14.804876 24950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:14.804913 24950 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 06:24:14.804939 24950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:14.810735 24950 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.131:40619
I20260926 06:24:14.810772 25077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.131:40619 every 8 connection(s)
I20260926 06:24:14.811069 24950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-2/data/info.pb
I20260926 06:24:14.815142 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 24950
I20260926 06:24:14.815274 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-2/wal/instance
I20260926 06:24:14.815516 4906 external_mini_cluster.cc:1428] Setting key d5a1723cf8b6feccd855fd205a6701fb
I20260926 06:24:14.816083 24671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.811728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:55649 (local address 127.4.202.190:34333)
0926 06:24:14.811796 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:14.811798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:14.812082 (+ 284us) server_negotiation.cc:407] Connection header received
0926 06:24:14.812519 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:14.812520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:14.812530 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:14.812579 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:14.813182 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.813658 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.814625 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.814839 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.814893 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:14.815793 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:14.815794 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:14.815794 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:14.815799 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:14.815815 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:14.815819 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:14.815882 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:14.815972 (+ 90us) server_negotiation.cc:299] Negotiation successful
0926 06:24:14.815999 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 06:24:14.816293 25080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.811727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:34333 (local address 127.4.202.131:55649)
0926 06:24:14.811932 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:14.811990 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:14.812440 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:14.812624 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:14.812627 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:14.812654 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:14.813113 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:14.813117 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.813731 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.813732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:14.814519 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:14.814521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.814910 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.814911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:14.815080 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:14.815640 (+ 560us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 06:24:14.815641 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:14.815677 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 06:24:14.815701 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:14.815903 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:14.815909 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:14.815912 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:24:14.815973 (+ 61us) client_negotiation.cc:240] Negotiation successful
0926 06:24:14.816071 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":99,"threads_started":1}
I20260926 06:24:14.816713 25078 heartbeater.cc:347] Connected to a master server at 127.4.202.190:34333
I20260926 06:24:14.816810 25078 heartbeater.cc:464] Registering TS with master...
I20260926 06:24:14.816936 25078 heartbeater.cc:511] Master 127.4.202.190:34333 requested a full tablet report, sending...
I20260926 06:24:14.817220 24628 ts_manager.cc:194] Registered new tserver with Master: 5dc2fe3dc4c745cf932d91b5274a37f3 (127.4.202.131:40619)
I20260926 06:24:14.817549 24628 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.202.131:55649
I20260926 06:24:14.827358 4906 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 06:24:14.831351 24671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.828592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48874 (local address 127.4.202.190:34333)
0926 06:24:14.828668 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:14.828669 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:14.828765 (+ 96us) server_negotiation.cc:407] Connection header received
0926 06:24:14.828835 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:14.828835 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:14.828848 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:14.828924 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:14.829088 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.829529 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.830257 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.830438 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.830515 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:14.831102 (+ 587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:14.831103 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:14.831103 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:14.831108 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:14.831119 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:14.831123 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:14.831169 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:14.831257 (+ 88us) server_negotiation.cc:299] Negotiation successful
0926 06:24:14.831278 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 06:24:14.832832 24628 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48874:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 06:24:14.834335 24628 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 06:24:14.840570 25086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.836423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.131:40619 (local address 127.0.0.1:41972)
0926 06:24:14.836554 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:14.836596 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:14.836664 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:14.836886 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:14.836889 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:14.836891 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:24:14.836948 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:14.836950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.838183 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.838185 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:14.839394 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:14.839396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.840280 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.840281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:14.840358 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:14.840371 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 06:24:14.840445 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 06:24:14.840503 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":64,"threads_started":1}
I20260926 06:24:14.840610 25087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.836498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41972 (local address 127.4.202.131:40619)
0926 06:24:14.836658 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:14.836663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:14.836673 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:24:14.836742 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:14.836744 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:14.836757 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:14.836823 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:24:14.837019 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.838095 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.839479 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.840184 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.840253 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:14.840334 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:14.840470 (+ 136us) server_negotiation.cc:299] Negotiation successful
0926 06:24:14.840533 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":83,"threads_started":1}
I20260926 06:24:14.840947 25091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.836984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58568 (local address 127.4.202.130:39879)
0926 06:24:14.837190 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:14.837194 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:14.837203 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:24:14.837238 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:14.837240 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:14.837254 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:14.837472 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:24:14.837527 (+ 55us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.838546 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.839735 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.840456 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.840517 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:14.840599 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:14.840807 (+ 208us) server_negotiation.cc:299] Negotiation successful
0926 06:24:14.840848 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":102,"threads_started":1}
I20260926 06:24:14.841056 25090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.836889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59692 (local address 127.4.202.129:35389)
0926 06:24:14.837122 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:14.837126 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:14.837283 (+ 157us) server_negotiation.cc:407] Connection header received
0926 06:24:14.837325 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:14.837327 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:14.837343 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:14.837413 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:24:14.837519 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.838198 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.839404 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.840195 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.840257 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:14.840334 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:14.840938 (+ 604us) server_negotiation.cc:299] Negotiation successful
0926 06:24:14.840970 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":119,"threads_started":1}
I20260926 06:24:14.840770 25088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.836614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:39879 (local address 127.0.0.1:58568)
0926 06:24:14.836738 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:14.836748 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:14.836793 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:14.837353 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:14.837354 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:14.837356 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:24:14.837406 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:14.837407 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.838640 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.838641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:14.839660 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:14.839662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.840536 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.840537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:14.840597 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:14.840602 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 06:24:14.840664 (+ 62us) client_negotiation.cc:240] Negotiation successful
0926 06:24:14.840707 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":75,"threads_started":1}
I20260926 06:24:14.841138 25089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.836784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:35389 (local address 127.0.0.1:59692)
0926 06:24:14.837173 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:14.837183 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:14.837228 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:14.837411 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:14.837412 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:14.837413 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:24:14.837465 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:14.837467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.838284 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.838285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:14.839322 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:14.839325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.840749 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.840750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:14.840829 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:14.840832 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:24:14.840920 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 06:24:14.840971 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":61,"threads_started":1}
I20260926 06:24:14.841889 25012 tablet_service.cc:1467] Processing CreateTablet for tablet e448cd309b3741138fcf6e4915e7f3d0 (DEFAULT_TABLE table=test-table [id=712cfc00fa5145d599e53e4337655db2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:24:14.841890 24750 tablet_service.cc:1467] Processing CreateTablet for tablet e448cd309b3741138fcf6e4915e7f3d0 (DEFAULT_TABLE table=test-table [id=712cfc00fa5145d599e53e4337655db2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:24:14.841991 24881 tablet_service.cc:1467] Processing CreateTablet for tablet e448cd309b3741138fcf6e4915e7f3d0 (DEFAULT_TABLE table=test-table [id=712cfc00fa5145d599e53e4337655db2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 06:24:14.842223 25012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e448cd309b3741138fcf6e4915e7f3d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:14.842223 24750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e448cd309b3741138fcf6e4915e7f3d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:14.842325 24881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e448cd309b3741138fcf6e4915e7f3d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:14.844820 25092 tablet_bootstrap.cc:492] T e448cd309b3741138fcf6e4915e7f3d0 P 9cd7ce62127e4b708d1721583f3b350e: Bootstrap starting.
I20260926 06:24:14.845306 25093 tablet_bootstrap.cc:492] T e448cd309b3741138fcf6e4915e7f3d0 P 5dc2fe3dc4c745cf932d91b5274a37f3: Bootstrap starting.
I20260926 06:24:14.845536 25092 tablet_bootstrap.cc:654] T e448cd309b3741138fcf6e4915e7f3d0 P 9cd7ce62127e4b708d1721583f3b350e: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:14.845873 25092 log.cc:824] T e448cd309b3741138fcf6e4915e7f3d0 P 9cd7ce62127e4b708d1721583f3b350e: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:14.845947 25093 tablet_bootstrap.cc:654] T e448cd309b3741138fcf6e4915e7f3d0 P 5dc2fe3dc4c745cf932d91b5274a37f3: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:14.846236 25093 log.cc:824] T e448cd309b3741138fcf6e4915e7f3d0 P 5dc2fe3dc4c745cf932d91b5274a37f3: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:14.846529 25094 tablet_bootstrap.cc:492] T e448cd309b3741138fcf6e4915e7f3d0 P 3280b77304e84a94b745365753cf400e: Bootstrap starting.
I20260926 06:24:14.846756 25092 tablet_bootstrap.cc:492] T e448cd309b3741138fcf6e4915e7f3d0 P 9cd7ce62127e4b708d1721583f3b350e: No bootstrap required, opened a new log
I20260926 06:24:14.846838 25092 ts_tablet_manager.cc:1397] T e448cd309b3741138fcf6e4915e7f3d0 P 9cd7ce62127e4b708d1721583f3b350e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 06:24:14.846938 25093 tablet_bootstrap.cc:492] T e448cd309b3741138fcf6e4915e7f3d0 P 5dc2fe3dc4c745cf932d91b5274a37f3: No bootstrap required, opened a new log
I20260926 06:24:14.847005 25093 ts_tablet_manager.cc:1397] T e448cd309b3741138fcf6e4915e7f3d0 P 5dc2fe3dc4c745cf932d91b5274a37f3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 06:24:14.847203 25094 tablet_bootstrap.cc:654] T e448cd309b3741138fcf6e4915e7f3d0 P 3280b77304e84a94b745365753cf400e: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:14.847528 25094 log.cc:824] T e448cd309b3741138fcf6e4915e7f3d0 P 3280b77304e84a94b745365753cf400e: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:14.848271 25094 tablet_bootstrap.cc:492] T e448cd309b3741138fcf6e4915e7f3d0 P 3280b77304e84a94b745365753cf400e: No bootstrap required, opened a new log
I20260926 06:24:14.848338 25094 ts_tablet_manager.cc:1397] T e448cd309b3741138fcf6e4915e7f3d0 P 3280b77304e84a94b745365753cf400e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 06:24:14.848953 25092 raft_consensus.cc:348] T e448cd309b3741138fcf6e4915e7f3d0 P 9cd7ce62127e4b708d1721583f3b350e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dc2fe3dc4c745cf932d91b5274a37f3" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40619 } } peers { permanent_uuid: "9cd7ce62127e4b708d1721583f3b350e" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 39879 } } peers { permanent_uuid: "3280b77304e84a94b745365753cf400e" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 35389 } }
I20260926 06:24:14.849102 25092 raft_consensus.cc:374] T e448cd309b3741138fcf6e4915e7f3d0 P 9cd7ce62127e4b708d1721583f3b350e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:14.849161 25092 raft_consensus.cc:729] T e448cd309b3741138fcf6e4915e7f3d0 P 9cd7ce62127e4b708d1721583f3b350e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cd7ce62127e4b708d1721583f3b350e, State: Initialized, Role: FOLLOWER
I20260926 06:24:14.849267 25092 consensus_queue.cc:260] T e448cd309b3741138fcf6e4915e7f3d0 P 9cd7ce62127e4b708d1721583f3b350e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dc2fe3dc4c745cf932d91b5274a37f3" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40619 } } peers { permanent_uuid: "9cd7ce62127e4b708d1721583f3b350e" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 39879 } } peers { permanent_uuid: "3280b77304e84a94b745365753cf400e" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 35389 } }
I20260926 06:24:14.849555 25092 ts_tablet_manager.cc:1428] T e448cd309b3741138fcf6e4915e7f3d0 P 9cd7ce62127e4b708d1721583f3b350e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 06:24:14.849593 24947 heartbeater.cc:503] Master 127.4.202.190:34333 was elected leader, sending a full tablet report...
I20260926 06:24:14.850319 25093 raft_consensus.cc:348] T e448cd309b3741138fcf6e4915e7f3d0 P 5dc2fe3dc4c745cf932d91b5274a37f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dc2fe3dc4c745cf932d91b5274a37f3" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40619 } } peers { permanent_uuid: "9cd7ce62127e4b708d1721583f3b350e" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 39879 } } peers { permanent_uuid: "3280b77304e84a94b745365753cf400e" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 35389 } }
I20260926 06:24:14.850498 25093 raft_consensus.cc:374] T e448cd309b3741138fcf6e4915e7f3d0 P 5dc2fe3dc4c745cf932d91b5274a37f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:14.850536 25093 raft_consensus.cc:729] T e448cd309b3741138fcf6e4915e7f3d0 P 5dc2fe3dc4c745cf932d91b5274a37f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dc2fe3dc4c745cf932d91b5274a37f3, State: Initialized, Role: FOLLOWER
I20260926 06:24:14.850663 25093 consensus_queue.cc:260] T e448cd309b3741138fcf6e4915e7f3d0 P 5dc2fe3dc4c745cf932d91b5274a37f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dc2fe3dc4c745cf932d91b5274a37f3" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40619 } } peers { permanent_uuid: "9cd7ce62127e4b708d1721583f3b350e" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 39879 } } peers { permanent_uuid: "3280b77304e84a94b745365753cf400e" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 35389 } }
I20260926 06:24:14.850726 25094 raft_consensus.cc:348] T e448cd309b3741138fcf6e4915e7f3d0 P 3280b77304e84a94b745365753cf400e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dc2fe3dc4c745cf932d91b5274a37f3" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40619 } } peers { permanent_uuid: "9cd7ce62127e4b708d1721583f3b350e" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 39879 } } peers { permanent_uuid: "3280b77304e84a94b745365753cf400e" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 35389 } }
I20260926 06:24:14.850879 25094 raft_consensus.cc:374] T e448cd309b3741138fcf6e4915e7f3d0 P 3280b77304e84a94b745365753cf400e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:14.850909 25093 ts_tablet_manager.cc:1428] T e448cd309b3741138fcf6e4915e7f3d0 P 5dc2fe3dc4c745cf932d91b5274a37f3: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260926 06:24:14.850924 25094 raft_consensus.cc:729] T e448cd309b3741138fcf6e4915e7f3d0 P 3280b77304e84a94b745365753cf400e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3280b77304e84a94b745365753cf400e, State: Initialized, Role: FOLLOWER
I20260926 06:24:14.850993 25078 heartbeater.cc:503] Master 127.4.202.190:34333 was elected leader, sending a full tablet report...
I20260926 06:24:14.851029 25094 consensus_queue.cc:260] T e448cd309b3741138fcf6e4915e7f3d0 P 3280b77304e84a94b745365753cf400e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dc2fe3dc4c745cf932d91b5274a37f3" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40619 } } peers { permanent_uuid: "9cd7ce62127e4b708d1721583f3b350e" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 39879 } } peers { permanent_uuid: "3280b77304e84a94b745365753cf400e" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 35389 } }
I20260926 06:24:14.851262 25094 ts_tablet_manager.cc:1428] T e448cd309b3741138fcf6e4915e7f3d0 P 3280b77304e84a94b745365753cf400e: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260926 06:24:14.851322 24816 heartbeater.cc:503] Master 127.4.202.190:34333 was elected leader, sending a full tablet report...
I20260926 06:24:14.917148 25099 raft_consensus.cc:482] T e448cd309b3741138fcf6e4915e7f3d0 P 5dc2fe3dc4c745cf932d91b5274a37f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 06:24:14.917254 25099 raft_consensus.cc:504] T e448cd309b3741138fcf6e4915e7f3d0 P 5dc2fe3dc4c745cf932d91b5274a37f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dc2fe3dc4c745cf932d91b5274a37f3" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40619 } } peers { permanent_uuid: "9cd7ce62127e4b708d1721583f3b350e" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 39879 } } peers { permanent_uuid: "3280b77304e84a94b745365753cf400e" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 35389 } }
I20260926 06:24:14.917536 25099 leader_election.cc:288] T e448cd309b3741138fcf6e4915e7f3d0 P 5dc2fe3dc4c745cf932d91b5274a37f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cd7ce62127e4b708d1721583f3b350e (127.4.202.130:39879), 3280b77304e84a94b745365753cf400e (127.4.202.129:35389)
I20260926 06:24:14.920316 25091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.917800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:59475 (local address 127.4.202.130:39879)
0926 06:24:14.917872 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:14.917874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:14.917883 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:24:14.917913 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:14.917914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:14.917923 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:14.918007 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:24:14.918158 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.918696 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.919624 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.919976 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.920059 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:14.920067 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:14.920218 (+ 151us) server_negotiation.cc:299] Negotiation successful
0926 06:24:14.920254 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 06:24:14.920316 25080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.917721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:39879 (local address 127.4.202.131:59475)
0926 06:24:14.917786 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:14.917799 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:14.917853 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:14.918013 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:14.918014 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:14.918016 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:24:14.918085 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:14.918087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.918769 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.918770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:14.919521 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:14.919523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.920057 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.920058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:14.920131 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:14.920134 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 06:24:14.920209 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 06:24:14.920247 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260926 06:24:14.920670 25090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.917953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.202.131:39213 (local address 127.4.202.129:35389)
0926 06:24:14.918011 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:14.918013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:14.918117 (+ 104us) server_negotiation.cc:407] Connection header received
0926 06:24:14.918184 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:14.918185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:14.918197 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:14.918269 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 06:24:14.918419 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.919016 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.919917 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.920288 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.920374 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:14.920382 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:14.920552 (+ 170us) server_negotiation.cc:299] Negotiation successful
0926 06:24:14.920596 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 06:24:14.920640 24901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e448cd309b3741138fcf6e4915e7f3d0" candidate_uuid: "5dc2fe3dc4c745cf932d91b5274a37f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cd7ce62127e4b708d1721583f3b350e" is_pre_election: true
I20260926 06:24:14.920789 24901 raft_consensus.cc:2454] T e448cd309b3741138fcf6e4915e7f3d0 P 9cd7ce62127e4b708d1721583f3b350e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5dc2fe3dc4c745cf932d91b5274a37f3 in term 0.
I20260926 06:24:14.920872 25101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.917864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:35389 (local address 127.4.202.131:39213)
0926 06:24:14.918035 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:14.918044 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:14.918126 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:14.918271 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:14.918272 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:14.918273 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 06:24:14.918342 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:14.918344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.919093 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.919095 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:14.919822 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:14.919824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.920382 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.920383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:14.920459 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:14.920463 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 06:24:14.920753 (+ 290us) client_negotiation.cc:240] Negotiation successful
0926 06:24:14.920792 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":106,"threads_started":1}
I20260926 06:24:14.921051 24965 leader_election.cc:302] T e448cd309b3741138fcf6e4915e7f3d0 P 5dc2fe3dc4c745cf932d91b5274a37f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5dc2fe3dc4c745cf932d91b5274a37f3, 9cd7ce62127e4b708d1721583f3b350e; no voters:
I20260926 06:24:14.921198 25099 raft_consensus.cc:2781] T e448cd309b3741138fcf6e4915e7f3d0 P 5dc2fe3dc4c745cf932d91b5274a37f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 06:24:14.921267 25099 raft_consensus.cc:482] T e448cd309b3741138fcf6e4915e7f3d0 P 5dc2fe3dc4c745cf932d91b5274a37f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 06:24:14.921299 25099 raft_consensus.cc:3037] T e448cd309b3741138fcf6e4915e7f3d0 P 5dc2fe3dc4c745cf932d91b5274a37f3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:14.921370 24770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e448cd309b3741138fcf6e4915e7f3d0" candidate_uuid: "5dc2fe3dc4c745cf932d91b5274a37f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3280b77304e84a94b745365753cf400e" is_pre_election: true
I20260926 06:24:14.921515 24770 raft_consensus.cc:2454] T e448cd309b3741138fcf6e4915e7f3d0 P 3280b77304e84a94b745365753cf400e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5dc2fe3dc4c745cf932d91b5274a37f3 in term 0.
I20260926 06:24:14.922050 25099 raft_consensus.cc:504] T e448cd309b3741138fcf6e4915e7f3d0 P 5dc2fe3dc4c745cf932d91b5274a37f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dc2fe3dc4c745cf932d91b5274a37f3" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40619 } } peers { permanent_uuid: "9cd7ce62127e4b708d1721583f3b350e" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 39879 } } peers { permanent_uuid: "3280b77304e84a94b745365753cf400e" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 35389 } }
I20260926 06:24:14.922181 25099 leader_election.cc:288] T e448cd309b3741138fcf6e4915e7f3d0 P 5dc2fe3dc4c745cf932d91b5274a37f3 [CANDIDATE]: Term 1 election: Requested vote from peers 9cd7ce62127e4b708d1721583f3b350e (127.4.202.130:39879), 3280b77304e84a94b745365753cf400e (127.4.202.129:35389)
I20260926 06:24:14.922314 24901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e448cd309b3741138fcf6e4915e7f3d0" candidate_uuid: "5dc2fe3dc4c745cf932d91b5274a37f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cd7ce62127e4b708d1721583f3b350e"
I20260926 06:24:14.922350 24770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e448cd309b3741138fcf6e4915e7f3d0" candidate_uuid: "5dc2fe3dc4c745cf932d91b5274a37f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3280b77304e84a94b745365753cf400e"
I20260926 06:24:14.922405 24901 raft_consensus.cc:3037] T e448cd309b3741138fcf6e4915e7f3d0 P 9cd7ce62127e4b708d1721583f3b350e [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:14.922446 24770 raft_consensus.cc:3037] T e448cd309b3741138fcf6e4915e7f3d0 P 3280b77304e84a94b745365753cf400e [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:14.923157 24770 raft_consensus.cc:2454] T e448cd309b3741138fcf6e4915e7f3d0 P 3280b77304e84a94b745365753cf400e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5dc2fe3dc4c745cf932d91b5274a37f3 in term 1.
I20260926 06:24:14.923250 24901 raft_consensus.cc:2454] T e448cd309b3741138fcf6e4915e7f3d0 P 9cd7ce62127e4b708d1721583f3b350e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5dc2fe3dc4c745cf932d91b5274a37f3 in term 1.
I20260926 06:24:14.923308 24965 leader_election.cc:302] T e448cd309b3741138fcf6e4915e7f3d0 P 5dc2fe3dc4c745cf932d91b5274a37f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3280b77304e84a94b745365753cf400e, 5dc2fe3dc4c745cf932d91b5274a37f3; no voters:
I20260926 06:24:14.923405 25099 raft_consensus.cc:2781] T e448cd309b3741138fcf6e4915e7f3d0 P 5dc2fe3dc4c745cf932d91b5274a37f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:24:14.923535 25099 raft_consensus.cc:686] T e448cd309b3741138fcf6e4915e7f3d0 P 5dc2fe3dc4c745cf932d91b5274a37f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5dc2fe3dc4c745cf932d91b5274a37f3, State: Running, Role: LEADER
I20260926 06:24:14.923605 25099 consensus_queue.cc:237] T e448cd309b3741138fcf6e4915e7f3d0 P 5dc2fe3dc4c745cf932d91b5274a37f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dc2fe3dc4c745cf932d91b5274a37f3" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40619 } } peers { permanent_uuid: "9cd7ce62127e4b708d1721583f3b350e" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 39879 } } peers { permanent_uuid: "3280b77304e84a94b745365753cf400e" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 35389 } }
I20260926 06:24:14.924340 24627 catalog_manager.cc:5495] T e448cd309b3741138fcf6e4915e7f3d0 P 5dc2fe3dc4c745cf932d91b5274a37f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5dc2fe3dc4c745cf932d91b5274a37f3 (127.4.202.131). New cstate: current_term: 1 leader_uuid: "5dc2fe3dc4c745cf932d91b5274a37f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dc2fe3dc4c745cf932d91b5274a37f3" member_type: VOTER last_known_addr { host: "127.4.202.131" port: 40619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9cd7ce62127e4b708d1721583f3b350e" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 39879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3280b77304e84a94b745365753cf400e" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 35389 } health_report { overall_health: UNKNOWN } } }
I20260926 06:24:14.931411 25087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.929048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41974 (local address 127.4.202.131:40619)
0926 06:24:14.929117 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:14.929119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:14.929128 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:24:14.929210 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:14.929210 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:14.929226 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:14.929291 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:24:14.929410 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.929849 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.930623 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:14.930843 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:14.930917 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:14.931100 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:24:14.931194 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:24:14.931284 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:14.931303 (+ 19us) server_negotiation.cc:299] Negotiation successful
0926 06:24:14.931319 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 06:24:14.932972 24901 raft_consensus.cc:1261] T e448cd309b3741138fcf6e4915e7f3d0 P 9cd7ce62127e4b708d1721583f3b350e [term 1 FOLLOWER]: Refusing update from remote peer 5dc2fe3dc4c745cf932d91b5274a37f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:24:14.932973 24770 raft_consensus.cc:1261] T e448cd309b3741138fcf6e4915e7f3d0 P 3280b77304e84a94b745365753cf400e [term 1 FOLLOWER]: Refusing update from remote peer 5dc2fe3dc4c745cf932d91b5274a37f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:24:14.933327 25102 consensus_queue.cc:1035] T e448cd309b3741138fcf6e4915e7f3d0 P 5dc2fe3dc4c745cf932d91b5274a37f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3280b77304e84a94b745365753cf400e" member_type: VOTER last_known_addr { host: "127.4.202.129" port: 35389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:24:14.933461 25099 consensus_queue.cc:1035] T e448cd309b3741138fcf6e4915e7f3d0 P 5dc2fe3dc4c745cf932d91b5274a37f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cd7ce62127e4b708d1721583f3b350e" member_type: VOTER last_known_addr { host: "127.4.202.130" port: 39879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:24:14.936522 25107 mvcc.cc:205] Tried to move back new op lower bound from 7333494189803044864 to 7333494189767704576. Current Snapshot: MvccSnapshot[applied={T|T < 7333494189803044864 or (T in {7333494189803044864})}]
I20260926 06:24:14.937949 24627 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:24:14.938060 24627 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 06:24:14.938740 24627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7107ae4085ee4140b09af771a576817e: Sending DeleteTablet for 3 replicas of tablet e448cd309b3741138fcf6e4915e7f3d0
I20260926 06:24:14.939033 25012 tablet_service.cc:1514] Processing DeleteTablet for tablet e448cd309b3741138fcf6e4915e7f3d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:24:14 UTC) from {username='slave'} at 127.0.0.1:41972
I20260926 06:24:14.939101 24881 tablet_service.cc:1514] Processing DeleteTablet for tablet e448cd309b3741138fcf6e4915e7f3d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:24:14 UTC) from {username='slave'} at 127.0.0.1:58568
I20260926 06:24:14.939164 24750 tablet_service.cc:1514] Processing DeleteTablet for tablet e448cd309b3741138fcf6e4915e7f3d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 06:24:14 UTC) from {username='slave'} at 127.0.0.1:59692
I20260926 06:24:14.939321 25115 tablet_replica.cc:330] T e448cd309b3741138fcf6e4915e7f3d0 P 5dc2fe3dc4c745cf932d91b5274a37f3: stopping tablet replica
I20260926 06:24:14.939416 25115 raft_consensus.cc:2229] T e448cd309b3741138fcf6e4915e7f3d0 P 5dc2fe3dc4c745cf932d91b5274a37f3 [term 1 LEADER]: Raft consensus shutting down.
I20260926 06:24:14.939507 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 24688
I20260926 06:24:14.939536 25115 raft_consensus.cc:2258] T e448cd309b3741138fcf6e4915e7f3d0 P 5dc2fe3dc4c745cf932d91b5274a37f3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 06:24:14.944787 24612 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.202.129:35389 (error 108)
I20260926 06:24:14.945024 25116 tablet_replica.cc:330] T e448cd309b3741138fcf6e4915e7f3d0 P 9cd7ce62127e4b708d1721583f3b350e: stopping tablet replica
I20260926 06:24:14.945037 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 24819
I20260926 06:24:14.945137 25116 raft_consensus.cc:2229] T e448cd309b3741138fcf6e4915e7f3d0 P 9cd7ce62127e4b708d1721583f3b350e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 06:24:14.945247 25116 raft_consensus.cc:2258] T e448cd309b3741138fcf6e4915e7f3d0 P 9cd7ce62127e4b708d1721583f3b350e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 06:24:14.946832 25115 ts_tablet_manager.cc:1905] T e448cd309b3741138fcf6e4915e7f3d0 P 5dc2fe3dc4c745cf932d91b5274a37f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 06:24:14.947263 25089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.947150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.129:35389 (local address 127.0.0.1:59706)
0926 06:24:14.947193 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:14.947230 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.129:35389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260926 06:24:14.947356 24612 catalog_manager.cc:4584] TS 3280b77304e84a94b745365753cf400e (127.4.202.129:35389): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e448cd309b3741138fcf6e4915e7f3d0: Network error: Client connection negotiation failed: client connection to 127.4.202.129:35389: connect: Connection refused (error 111)
I20260926 06:24:14.948518 25115 ts_tablet_manager.cc:1918] T e448cd309b3741138fcf6e4915e7f3d0 P 5dc2fe3dc4c745cf932d91b5274a37f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 06:24:14.948601 25115 log.cc:1196] T e448cd309b3741138fcf6e4915e7f3d0 P 5dc2fe3dc4c745cf932d91b5274a37f3: Deleting WAL directory at /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790403591640726-4906-0/minicluster-data/ts-2/wal/wals/e448cd309b3741138fcf6e4915e7f3d0
I20260926 06:24:14.948827 25115 ts_tablet_manager.cc:1939] T e448cd309b3741138fcf6e4915e7f3d0 P 5dc2fe3dc4c745cf932d91b5274a37f3: Deleting consensus metadata
I20260926 06:24:14.949208 24612 catalog_manager.cc:4841] TS 5dc2fe3dc4c745cf932d91b5274a37f3 (127.4.202.131:40619): tablet e448cd309b3741138fcf6e4915e7f3d0 (table test-table [id=712cfc00fa5145d599e53e4337655db2]) successfully deleted
I20260926 06:24:14.950888 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 24950
I20260926 06:24:14.950903 25089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:14.950794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.130:39879 (local address 127.0.0.1:58570)
0926 06:24:14.950851 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:14.950882 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.130:39879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260926 06:24:14.956456 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 24597
2026-09-26T06:24:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (677 ms)
[----------] 6 tests from EncryptionPolicyTest (8315 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:24:15 dist-test-slave-pd0t krb5kdc[25121](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:24:15 dist-test-slave-pd0t krb5kdc[25121](info): set up 2 sockets
Sep 26 06:24:15 dist-test-slave-pd0t krb5kdc[25121](info): commencing operation
krb5kdc: starting...
W20260926 06:24:15.978542 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:24:16 dist-test-slave-pd0t krb5kdc[25121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:24:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:24:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 06:24:16.089771 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33773
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:24:16.169334 25134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:16.169548 25134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:16.169584 25134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:16.170729 25134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:24:16.170784 25134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:16.170806 25134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:16.170826 25134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:24:16.170853 25134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:24:16.172271 25134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33773
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:16.172529 25134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:16.172746 25134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:16.177343 25142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:16.177349 25140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:16.177369 25139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:16.177661 25134 server_base.cc:1034] running on GCE node
I20260926 06:24:16.177872 25134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:16.178158 25134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:16.179315 25134 hybrid_clock.cc:648] HybridClock initialized: now 1790403856179303 us; error 35 us; skew 500 ppm
Sep 26 06:24:16 dist-test-slave-pd0t krb5kdc[25121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403856, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:16.181739 25134 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 06:24:16.182157 25134 webserver.cc:460] Webserver started at http://127.0.0.1:41891/ using document root <none> and password file <none>
I20260926 06:24:16.182354 25134 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:16.182415 25134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:16.182550 25134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:16.183458 25134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "e7cf3bc9314a4d9fa1690e51aefd083d"
format_stamp: "Formatted at 2026-09-26 06:24:16 on dist-test-slave-pd0t"
server_key: "0f3f36cbbc1e66f3344a67e5f41c6e22"
server_key_iv: "1cce12e60867273c4d437bf2271369a1"
server_key_version: "encryptionkey@0"
I20260926 06:24:16.183781 25134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "e7cf3bc9314a4d9fa1690e51aefd083d"
format_stamp: "Formatted at 2026-09-26 06:24:16 on dist-test-slave-pd0t"
server_key: "0f3f36cbbc1e66f3344a67e5f41c6e22"
server_key_iv: "1cce12e60867273c4d437bf2271369a1"
server_key_version: "encryptionkey@0"
I20260926 06:24:16.185087 25134 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:24:16.185770 25149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:16.185966 25134 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:24:16.186062 25134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "e7cf3bc9314a4d9fa1690e51aefd083d"
format_stamp: "Formatted at 2026-09-26 06:24:16 on dist-test-slave-pd0t"
server_key: "0f3f36cbbc1e66f3344a67e5f41c6e22"
server_key_iv: "1cce12e60867273c4d437bf2271369a1"
server_key_version: "encryptionkey@0"
I20260926 06:24:16.186139 25134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:16.211287 25134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:16.213506 25134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:16.213651 25134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:16.217710 25134 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33773
I20260926 06:24:16.217741 25201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33773 every 8 connection(s)
I20260926 06:24:16.218057 25134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:24:16.218941 25202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:16.221056 25202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7cf3bc9314a4d9fa1690e51aefd083d: Bootstrap starting.
I20260926 06:24:16.221592 25202 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7cf3bc9314a4d9fa1690e51aefd083d: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:16.221870 25202 log.cc:824] T 00000000000000000000000000000000 P e7cf3bc9314a4d9fa1690e51aefd083d: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:16.222505 25202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7cf3bc9314a4d9fa1690e51aefd083d: No bootstrap required, opened a new log
I20260926 06:24:16.224114 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 25134
I20260926 06:24:16.224071 25202 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7cf3bc9314a4d9fa1690e51aefd083d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7cf3bc9314a4d9fa1690e51aefd083d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33773 } }
I20260926 06:24:16.224181 25202 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7cf3bc9314a4d9fa1690e51aefd083d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:16.224202 25202 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7cf3bc9314a4d9fa1690e51aefd083d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7cf3bc9314a4d9fa1690e51aefd083d, State: Initialized, Role: FOLLOWER
I20260926 06:24:16.224237 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:24:16.224360 25202 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7cf3bc9314a4d9fa1690e51aefd083d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7cf3bc9314a4d9fa1690e51aefd083d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33773 } }
I20260926 06:24:16.224428 25202 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7cf3bc9314a4d9fa1690e51aefd083d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:24:16.224453 4906 external_mini_cluster.cc:1428] Setting key 25151ce196344cd91e604dcfde364408
I20260926 06:24:16.224472 25202 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7cf3bc9314a4d9fa1690e51aefd083d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:24:16.224514 25202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7cf3bc9314a4d9fa1690e51aefd083d [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:16.225518 25202 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7cf3bc9314a4d9fa1690e51aefd083d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7cf3bc9314a4d9fa1690e51aefd083d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33773 } }
I20260926 06:24:16.225646 25202 leader_election.cc:302] T 00000000000000000000000000000000 P e7cf3bc9314a4d9fa1690e51aefd083d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7cf3bc9314a4d9fa1690e51aefd083d; no voters:
I20260926 06:24:16.225812 25202 leader_election.cc:288] T 00000000000000000000000000000000 P e7cf3bc9314a4d9fa1690e51aefd083d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:24:16.225869 25207 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7cf3bc9314a4d9fa1690e51aefd083d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:24:16.226030 25207 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7cf3bc9314a4d9fa1690e51aefd083d [term 1 LEADER]: Becoming Leader. State: Replica: e7cf3bc9314a4d9fa1690e51aefd083d, State: Running, Role: LEADER
I20260926 06:24:16.226068 25202 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7cf3bc9314a4d9fa1690e51aefd083d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:24:16.226109 25207 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7cf3bc9314a4d9fa1690e51aefd083d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7cf3bc9314a4d9fa1690e51aefd083d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33773 } }
I20260926 06:24:16.226392 25207 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7cf3bc9314a4d9fa1690e51aefd083d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7cf3bc9314a4d9fa1690e51aefd083d. Latest consensus state: current_term: 1 leader_uuid: "e7cf3bc9314a4d9fa1690e51aefd083d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7cf3bc9314a4d9fa1690e51aefd083d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33773 } } }
I20260926 06:24:16.226449 25207 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7cf3bc9314a4d9fa1690e51aefd083d [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:16.226568 25208 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7cf3bc9314a4d9fa1690e51aefd083d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7cf3bc9314a4d9fa1690e51aefd083d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7cf3bc9314a4d9fa1690e51aefd083d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33773 } } }
I20260926 06:24:16.226665 25208 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7cf3bc9314a4d9fa1690e51aefd083d [sys.catalog]: This master's current role is: LEADER
W20260926 06:24:16.228091 25222 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e7cf3bc9314a4d9fa1690e51aefd083d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:24:16.228192 25222 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:24:16.228255 25223 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:24:16.228391 25223 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:24:16.229784 25223 catalog_manager.cc:1347] Generated new cluster ID: 84c3ad9e283d451b8d8d1d028bea97b0
I20260926 06:24:16.229827 25223 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:24:16 dist-test-slave-pd0t krb5kdc[25121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 06:24:16.235903 25206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:16.224945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57699 (local address 127.0.0.1:33773)
0926 06:24:16.225172 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:16.225180 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:16.225202 (+ 22us) server_negotiation.cc:407] Connection header received
0926 06:24:16.225284 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:16.225288 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:16.225325 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:16.225425 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:16.226298 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:16.227937 (+ 1639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:16.228914 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:16.229420 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:16.229526 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:16.232260 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:16.232263 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:16.232264 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:16.232275 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:16.234209 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:16.234660 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:16.234663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:16.234664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:16.234804 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:16.235041 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:16.235042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:16.235043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:16.235341 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:16.235518 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:16.235599 (+ 81us) server_negotiation.cc:299] Negotiation successful
0926 06:24:16.235699 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":127,"threads_started":1}
I20260926 06:24:16.236900 4906 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:24:16.243716 25223 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:24:16.244143 25223 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:24:16.244831 25206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:16.238270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33183 (local address 127.0.0.1:33773)
0926 06:24:16.238311 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:16.238313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:16.238342 (+ 29us) server_negotiation.cc:407] Connection header received
0926 06:24:16.238359 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:16.238360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:16.238368 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:16.238419 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:16.238978 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:16.239704 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:16.240572 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:16.240893 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:16.240966 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:16.242157 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:16.242159 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:16.242159 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:16.242166 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:16.243574 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:16.244013 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:16.244014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:16.244014 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:16.244061 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:16.244270 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:16.244271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:16.244271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:16.244371 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:16.244447 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:16.244675 (+ 228us) server_negotiation.cc:299] Negotiation successful
0926 06:24:16.244750 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 06:24:16.249645 25223 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7cf3bc9314a4d9fa1690e51aefd083d: Generated new TSK 0
I20260926 06:24:16.249804 25223 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:24:16.256218 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 25134
W20260926 06:24:16.260423 25119 connection.cc:535] client connection to 127.0.0.1:33773 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33773): Connection reset by peer (error 104)
2026-09-26T06:24:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1298 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:24:16 dist-test-slave-pd0t krb5kdc[25233](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:24:16 dist-test-slave-pd0t krb5kdc[25233](info): set up 2 sockets
Sep 26 06:24:16 dist-test-slave-pd0t krb5kdc[25233](info): commencing operation
krb5kdc: starting...
W20260926 06:24:17.278543 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:24:17 dist-test-slave-pd0t krb5kdc[25233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:24:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:24:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 06:24:17.392827 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35559
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:24:17.471357 25246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:17.471547 25246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:17.471570 25246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:17.472702 25246 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:24:17.472743 25246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:17.472756 25246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:17.472810 25246 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:24:17.472837 25246 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:24:17.474285 25246 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35559
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:17.474525 25246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:17.474768 25246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:17.479342 25251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:17.479389 25254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:17.479372 25252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:17.479532 25246 server_base.cc:1034] running on GCE node
I20260926 06:24:17.479796 25246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:17.480074 25246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:17.481235 25246 hybrid_clock.cc:648] HybridClock initialized: now 1790403857481227 us; error 34 us; skew 500 ppm
Sep 26 06:24:17 dist-test-slave-pd0t krb5kdc[25233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403857, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:17.489058 25246 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 06:24:17.489471 25246 webserver.cc:460] Webserver started at http://127.0.0.1:36311/ using document root <none> and password file <none>
I20260926 06:24:17.489696 25246 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:17.489758 25246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:17.489897 25246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:17.490981 25246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "4410b57b1fa94da9a35d9b282d2a20f3"
format_stamp: "Formatted at 2026-09-26 06:24:17 on dist-test-slave-pd0t"
server_key: "b7143149b729bbe66090c0ba748738e8"
server_key_iv: "b815c2ee86262b7b11b58af455361cf8"
server_key_version: "encryptionkey@0"
I20260926 06:24:17.491323 25246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "4410b57b1fa94da9a35d9b282d2a20f3"
format_stamp: "Formatted at 2026-09-26 06:24:17 on dist-test-slave-pd0t"
server_key: "b7143149b729bbe66090c0ba748738e8"
server_key_iv: "b815c2ee86262b7b11b58af455361cf8"
server_key_version: "encryptionkey@0"
I20260926 06:24:17.492610 25246 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:24:17.493384 25261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:17.493597 25246 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:24:17.493695 25246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "4410b57b1fa94da9a35d9b282d2a20f3"
format_stamp: "Formatted at 2026-09-26 06:24:17 on dist-test-slave-pd0t"
server_key: "b7143149b729bbe66090c0ba748738e8"
server_key_iv: "b815c2ee86262b7b11b58af455361cf8"
server_key_version: "encryptionkey@0"
I20260926 06:24:17.493762 25246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:17.512632 25246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:17.514549 25246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:17.514724 25246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:17.518462 25246 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35559
I20260926 06:24:17.518496 25313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35559 every 8 connection(s)
I20260926 06:24:17.518836 25246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:24:17.519739 25314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:17.521889 25314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4410b57b1fa94da9a35d9b282d2a20f3: Bootstrap starting.
I20260926 06:24:17.522460 25314 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4410b57b1fa94da9a35d9b282d2a20f3: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:17.522749 25314 log.cc:824] T 00000000000000000000000000000000 P 4410b57b1fa94da9a35d9b282d2a20f3: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:17.523422 25314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4410b57b1fa94da9a35d9b282d2a20f3: No bootstrap required, opened a new log
I20260926 06:24:17.524907 25314 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4410b57b1fa94da9a35d9b282d2a20f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4410b57b1fa94da9a35d9b282d2a20f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35559 } }
I20260926 06:24:17.525038 25314 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4410b57b1fa94da9a35d9b282d2a20f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:17.525092 25314 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4410b57b1fa94da9a35d9b282d2a20f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4410b57b1fa94da9a35d9b282d2a20f3, State: Initialized, Role: FOLLOWER
I20260926 06:24:17.525228 25314 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4410b57b1fa94da9a35d9b282d2a20f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4410b57b1fa94da9a35d9b282d2a20f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35559 } }
I20260926 06:24:17.525287 25314 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4410b57b1fa94da9a35d9b282d2a20f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:24:17.525308 25314 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4410b57b1fa94da9a35d9b282d2a20f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:24:17.525373 25314 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4410b57b1fa94da9a35d9b282d2a20f3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:17.525882 25314 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4410b57b1fa94da9a35d9b282d2a20f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4410b57b1fa94da9a35d9b282d2a20f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35559 } }
I20260926 06:24:17.525992 25314 leader_election.cc:302] T 00000000000000000000000000000000 P 4410b57b1fa94da9a35d9b282d2a20f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4410b57b1fa94da9a35d9b282d2a20f3; no voters:
I20260926 06:24:17.526163 25314 leader_election.cc:288] T 00000000000000000000000000000000 P 4410b57b1fa94da9a35d9b282d2a20f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:24:17.526189 25317 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4410b57b1fa94da9a35d9b282d2a20f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:24:17.526424 25314 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4410b57b1fa94da9a35d9b282d2a20f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:24:17.526453 25317 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4410b57b1fa94da9a35d9b282d2a20f3 [term 1 LEADER]: Becoming Leader. State: Replica: 4410b57b1fa94da9a35d9b282d2a20f3, State: Running, Role: LEADER
I20260926 06:24:17.526530 25317 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4410b57b1fa94da9a35d9b282d2a20f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4410b57b1fa94da9a35d9b282d2a20f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35559 } }
I20260926 06:24:17.527052 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 25246
I20260926 06:24:17.527045 25319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4410b57b1fa94da9a35d9b282d2a20f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4410b57b1fa94da9a35d9b282d2a20f3. Latest consensus state: current_term: 1 leader_uuid: "4410b57b1fa94da9a35d9b282d2a20f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4410b57b1fa94da9a35d9b282d2a20f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35559 } } }
I20260926 06:24:17.527120 25319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4410b57b1fa94da9a35d9b282d2a20f3 [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:17.527177 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:24:17.527314 25326 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:24:17.527381 4906 external_mini_cluster.cc:1428] Setting key 9d3e1b639d0391cc4abaea905ead12c2
I20260926 06:24:17.527443 25318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4410b57b1fa94da9a35d9b282d2a20f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4410b57b1fa94da9a35d9b282d2a20f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4410b57b1fa94da9a35d9b282d2a20f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35559 } } }
I20260926 06:24:17.527504 25318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4410b57b1fa94da9a35d9b282d2a20f3 [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:17.527815 25326 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:24:17.529294 25326 catalog_manager.cc:1347] Generated new cluster ID: 56acf61d9d1a4f74b29fec09b20eefd7
I20260926 06:24:17.529343 25326 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:24:17.533034 25335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:17.527982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43241 (local address 127.0.0.1:35559)
0926 06:24:17.528331 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:17.528341 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:17.528561 (+ 220us) server_negotiation.cc:407] Connection header received
0926 06:24:17.528681 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:17.528687 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:17.528740 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:17.528881 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:17.529085 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:17.530144 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:17.531148 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:17.531576 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:17.531695 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:17.532406 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:17.532417 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:17.532418 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:17.532587 (+ 169us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:17.532616 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:17.532626 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:17.532720 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:17.532800 (+ 80us) server_negotiation.cc:299] Negotiation successful
0926 06:24:17.532829 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":52,"threads_started":1}
I20260926 06:24:17.533872 4906 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:24:17.541364 25335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:17.535238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53809 (local address 127.0.0.1:35559)
0926 06:24:17.535276 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:17.535278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:17.538760 (+ 3482us) server_negotiation.cc:407] Connection header received
0926 06:24:17.538831 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:17.538832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:17.538843 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:17.538891 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:17.539011 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:17.539476 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:17.540176 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:17.540417 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:17.540492 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:17.541128 (+ 636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:17.541130 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:17.541130 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:17.541135 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:17.541146 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:17.541150 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:17.541203 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:17.541262 (+ 59us) server_negotiation.cc:299] Negotiation successful
0926 06:24:17.541274 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 06:24:17.542739 25326 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:24:17.543205 25326 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:24:17.549691 25326 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4410b57b1fa94da9a35d9b282d2a20f3: Generated new TSK 0
I20260926 06:24:17.549881 25326 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:24:17.554740 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 25246
W20260926 06:24:17.558836 25231 connection.cc:535] client connection to 127.0.0.1:35559 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35559): Connection reset by peer (error 104)
2026-09-26T06:24:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1298 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:24:17 dist-test-slave-pd0t krb5kdc[25345](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:24:17 dist-test-slave-pd0t krb5kdc[25345](info): set up 2 sockets
Sep 26 06:24:17 dist-test-slave-pd0t krb5kdc[25345](info): commencing operation
krb5kdc: starting...
W20260926 06:24:18.590775 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:24:18 dist-test-slave-pd0t krb5kdc[25345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:24:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:24:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 06:24:18.701453 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37641
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:24:18.778486 25358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:18.778688 25358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:18.778712 25358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:18.779815 25358 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:24:18.779856 25358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:18.779870 25358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:18.779881 25358 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:24:18.779946 25358 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:24:18.781420 25358 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37641
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:18.781697 25358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:18.781915 25358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 06:24:18.786826 25358 server_base.cc:1034] running on GCE node
W20260926 06:24:18.786795 25366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:18.786790 25364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:18.786782 25363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:18.787292 25358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:18.787565 25358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:18.788735 25358 hybrid_clock.cc:648] HybridClock initialized: now 1790403858788716 us; error 35 us; skew 500 ppm
Sep 26 06:24:18 dist-test-slave-pd0t krb5kdc[25345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403858, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:18.791127 25358 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 06:24:18.791478 25358 webserver.cc:460] Webserver started at http://127.0.0.1:45037/ using document root <none> and password file <none>
I20260926 06:24:18.791652 25358 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:18.791690 25358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:18.791770 25358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:18.792737 25358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "5e63785d230643a9a07cb4dd47a8bd30"
format_stamp: "Formatted at 2026-09-26 06:24:18 on dist-test-slave-pd0t"
server_key: "83bb6e2692de0ab21401e051fa96c1a4"
server_key_iv: "00742fb74947d817e0c69f1c625def45"
server_key_version: "encryptionkey@0"
I20260926 06:24:18.793061 25358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "5e63785d230643a9a07cb4dd47a8bd30"
format_stamp: "Formatted at 2026-09-26 06:24:18 on dist-test-slave-pd0t"
server_key: "83bb6e2692de0ab21401e051fa96c1a4"
server_key_iv: "00742fb74947d817e0c69f1c625def45"
server_key_version: "encryptionkey@0"
I20260926 06:24:18.794216 25358 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 06:24:18.794919 25373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:18.795192 25358 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:24:18.795272 25358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "5e63785d230643a9a07cb4dd47a8bd30"
format_stamp: "Formatted at 2026-09-26 06:24:18 on dist-test-slave-pd0t"
server_key: "83bb6e2692de0ab21401e051fa96c1a4"
server_key_iv: "00742fb74947d817e0c69f1c625def45"
server_key_version: "encryptionkey@0"
I20260926 06:24:18.795348 25358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:18.798449 25358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:18.800248 25358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:18.800395 25358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:18.804051 25358 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37641
I20260926 06:24:18.804117 25425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37641 every 8 connection(s)
I20260926 06:24:18.804400 25358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:24:18.805301 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 25358
I20260926 06:24:18.805384 25426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:18.805423 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:24:18.805629 4906 external_mini_cluster.cc:1428] Setting key a991440cb8f420983e2bca7bd0bceb8e
I20260926 06:24:18.808142 25429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:18.806113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60477 (local address 127.0.0.1:37641)
0926 06:24:18.806338 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:18.806347 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:18.806380 (+ 33us) server_negotiation.cc:407] Connection header received
0926 06:24:18.806928 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:18.806936 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:18.806970 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:18.807116 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:18.807728 (+ 612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:18.807731 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:18.807732 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:18.807736 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:18.807782 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:18.807796 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:18.807874 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:18.807933 (+ 59us) server_negotiation.cc:299] Negotiation successful
0926 06:24:18.807971 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":74,"threads_started":1}
I20260926 06:24:18.808605 25426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e63785d230643a9a07cb4dd47a8bd30: Bootstrap starting.
I20260926 06:24:18.809177 4906 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:24:18.809305 25426 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e63785d230643a9a07cb4dd47a8bd30: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:18.809579 25426 log.cc:824] T 00000000000000000000000000000000 P 5e63785d230643a9a07cb4dd47a8bd30: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:18.810329 25426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e63785d230643a9a07cb4dd47a8bd30: No bootstrap required, opened a new log
I20260926 06:24:18.811781 25429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:18.810763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37683 (local address 127.0.0.1:37641)
0926 06:24:18.810795 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:18.810797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:18.810954 (+ 157us) server_negotiation.cc:407] Connection header received
0926 06:24:18.811008 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:18.811009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:18.811013 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:18.811059 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:18.811575 (+ 516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:18.811576 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:18.811576 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:18.811579 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:18.811590 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:18.811593 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:18.811632 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:18.811692 (+ 60us) server_negotiation.cc:299] Negotiation successful
0926 06:24:18.811701 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 06:24:18.812845 25426 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e63785d230643a9a07cb4dd47a8bd30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e63785d230643a9a07cb4dd47a8bd30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37641 } }
I20260926 06:24:18.813009 25426 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e63785d230643a9a07cb4dd47a8bd30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:18.813056 25426 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e63785d230643a9a07cb4dd47a8bd30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e63785d230643a9a07cb4dd47a8bd30, State: Initialized, Role: FOLLOWER
I20260926 06:24:18.813169 25426 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e63785d230643a9a07cb4dd47a8bd30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e63785d230643a9a07cb4dd47a8bd30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37641 } }
I20260926 06:24:18.813273 25426 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e63785d230643a9a07cb4dd47a8bd30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:24:18.813316 25426 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e63785d230643a9a07cb4dd47a8bd30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:24:18.813365 25426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e63785d230643a9a07cb4dd47a8bd30 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:18.813853 25426 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e63785d230643a9a07cb4dd47a8bd30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e63785d230643a9a07cb4dd47a8bd30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37641 } }
I20260926 06:24:18.813971 25426 leader_election.cc:302] T 00000000000000000000000000000000 P 5e63785d230643a9a07cb4dd47a8bd30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e63785d230643a9a07cb4dd47a8bd30; no voters:
I20260926 06:24:18.814158 25426 leader_election.cc:288] T 00000000000000000000000000000000 P 5e63785d230643a9a07cb4dd47a8bd30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:24:18.814186 25436 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e63785d230643a9a07cb4dd47a8bd30 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:24:18.814391 25426 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e63785d230643a9a07cb4dd47a8bd30 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:24:18.814419 25436 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e63785d230643a9a07cb4dd47a8bd30 [term 1 LEADER]: Becoming Leader. State: Replica: 5e63785d230643a9a07cb4dd47a8bd30, State: Running, Role: LEADER
I20260926 06:24:18.814525 25436 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e63785d230643a9a07cb4dd47a8bd30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e63785d230643a9a07cb4dd47a8bd30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37641 } }
I20260926 06:24:18.814916 25437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e63785d230643a9a07cb4dd47a8bd30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e63785d230643a9a07cb4dd47a8bd30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e63785d230643a9a07cb4dd47a8bd30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37641 } } }
I20260926 06:24:18.814895 25438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e63785d230643a9a07cb4dd47a8bd30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e63785d230643a9a07cb4dd47a8bd30. Latest consensus state: current_term: 1 leader_uuid: "5e63785d230643a9a07cb4dd47a8bd30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e63785d230643a9a07cb4dd47a8bd30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37641 } } }
I20260926 06:24:18.815006 25437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e63785d230643a9a07cb4dd47a8bd30 [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:18.815014 25438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e63785d230643a9a07cb4dd47a8bd30 [sys.catalog]: This master's current role is: LEADER
W20260926 06:24:18.816213 25452 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5e63785d230643a9a07cb4dd47a8bd30: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:24:18.816256 25452 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:24:18.816311 25444 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:24:18.816473 25444 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:24:18.817857 25444 catalog_manager.cc:1347] Generated new cluster ID: 83d769700ee84f54adf45d93f93a65b2
I20260926 06:24:18.817934 25444 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:24:18.830379 25444 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:24:18.830533 25444 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:24:18.835305 25444 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e63785d230643a9a07cb4dd47a8bd30: Generated new TSK 0
I20260926 06:24:18.835465 25444 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:24:18.840689 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 25358
2026-09-26T06:24:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1285 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:24:18 dist-test-slave-pd0t krb5kdc[25457](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:24:18 dist-test-slave-pd0t krb5kdc[25457](info): set up 2 sockets
Sep 26 06:24:18 dist-test-slave-pd0t krb5kdc[25457](info): commencing operation
krb5kdc: starting...
W20260926 06:24:19.862344 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:24:19 dist-test-slave-pd0t krb5kdc[25457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:24:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:24:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 06:24:19.973665 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35789
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35789
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:24:20.050901 25470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:20.051086 25470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:20.051107 25470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:20.052260 25470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:24:20.052302 25470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:20.052316 25470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:20.052328 25470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:24:20.052340 25470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:24:20.053823 25470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35789
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:20.054085 25470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:20.054311 25470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:20.058997 25478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:20.059026 25475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:20.059067 25470 server_base.cc:1034] running on GCE node
W20260926 06:24:20.059037 25476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:20.059415 25470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:20.059682 25470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:20.060851 25470 hybrid_clock.cc:648] HybridClock initialized: now 1790403860060840 us; error 42 us; skew 500 ppm
Sep 26 06:24:20 dist-test-slave-pd0t krb5kdc[25457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403860, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:20.063340 25470 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 06:24:20.063730 25470 webserver.cc:460] Webserver started at http://127.0.0.1:33095/ using document root <none> and password file <none>
I20260926 06:24:20.063939 25470 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:20.064003 25470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:20.064167 25470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:20.065157 25470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "77e98fde2d274aba9e350674d8e7c06f"
format_stamp: "Formatted at 2026-09-26 06:24:20 on dist-test-slave-pd0t"
server_key: "ad73a04fea37c6c291a1659c9fa8a190"
server_key_iv: "41282b37058404d7d3b624a13d095d17"
server_key_version: "encryptionkey@0"
I20260926 06:24:20.065505 25470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "77e98fde2d274aba9e350674d8e7c06f"
format_stamp: "Formatted at 2026-09-26 06:24:20 on dist-test-slave-pd0t"
server_key: "ad73a04fea37c6c291a1659c9fa8a190"
server_key_iv: "41282b37058404d7d3b624a13d095d17"
server_key_version: "encryptionkey@0"
I20260926 06:24:20.066807 25470 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 06:24:20.067577 25485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:20.067756 25470 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:24:20.067852 25470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "77e98fde2d274aba9e350674d8e7c06f"
format_stamp: "Formatted at 2026-09-26 06:24:20 on dist-test-slave-pd0t"
server_key: "ad73a04fea37c6c291a1659c9fa8a190"
server_key_iv: "41282b37058404d7d3b624a13d095d17"
server_key_version: "encryptionkey@0"
I20260926 06:24:20.067927 25470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:20.082084 25470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:20.084307 25470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:20.084465 25470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:20.088364 25470 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35789
I20260926 06:24:20.088402 25537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35789 every 8 connection(s)
I20260926 06:24:20.088726 25470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:24:20.089617 25538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:20.091780 25538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77e98fde2d274aba9e350674d8e7c06f: Bootstrap starting.
I20260926 06:24:20.092372 25538 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77e98fde2d274aba9e350674d8e7c06f: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:20.092653 25538 log.cc:824] T 00000000000000000000000000000000 P 77e98fde2d274aba9e350674d8e7c06f: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:20.093325 25538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77e98fde2d274aba9e350674d8e7c06f: No bootstrap required, opened a new log
I20260926 06:24:20.094928 25538 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77e98fde2d274aba9e350674d8e7c06f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77e98fde2d274aba9e350674d8e7c06f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35789 } }
I20260926 06:24:20.095055 25538 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77e98fde2d274aba9e350674d8e7c06f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:20.095077 25538 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77e98fde2d274aba9e350674d8e7c06f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77e98fde2d274aba9e350674d8e7c06f, State: Initialized, Role: FOLLOWER
I20260926 06:24:20.095222 25538 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77e98fde2d274aba9e350674d8e7c06f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77e98fde2d274aba9e350674d8e7c06f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35789 } }
I20260926 06:24:20.095284 25538 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77e98fde2d274aba9e350674d8e7c06f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:24:20.095305 25538 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77e98fde2d274aba9e350674d8e7c06f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:24:20.095358 25538 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77e98fde2d274aba9e350674d8e7c06f [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:20.095872 25538 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77e98fde2d274aba9e350674d8e7c06f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77e98fde2d274aba9e350674d8e7c06f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35789 } }
I20260926 06:24:20.095980 25538 leader_election.cc:302] T 00000000000000000000000000000000 P 77e98fde2d274aba9e350674d8e7c06f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77e98fde2d274aba9e350674d8e7c06f; no voters:
I20260926 06:24:20.096155 25538 leader_election.cc:288] T 00000000000000000000000000000000 P 77e98fde2d274aba9e350674d8e7c06f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:24:20.096215 25541 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77e98fde2d274aba9e350674d8e7c06f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:24:20.096380 25541 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77e98fde2d274aba9e350674d8e7c06f [term 1 LEADER]: Becoming Leader. State: Replica: 77e98fde2d274aba9e350674d8e7c06f, State: Running, Role: LEADER
I20260926 06:24:20.096402 25538 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77e98fde2d274aba9e350674d8e7c06f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:24:20.096510 25541 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77e98fde2d274aba9e350674d8e7c06f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77e98fde2d274aba9e350674d8e7c06f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35789 } }
I20260926 06:24:20.096879 25542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77e98fde2d274aba9e350674d8e7c06f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77e98fde2d274aba9e350674d8e7c06f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77e98fde2d274aba9e350674d8e7c06f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35789 } } }
I20260926 06:24:20.096882 25543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77e98fde2d274aba9e350674d8e7c06f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77e98fde2d274aba9e350674d8e7c06f. Latest consensus state: current_term: 1 leader_uuid: "77e98fde2d274aba9e350674d8e7c06f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77e98fde2d274aba9e350674d8e7c06f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35789 } } }
I20260926 06:24:20.097047 25542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77e98fde2d274aba9e350674d8e7c06f [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:20.097070 25543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77e98fde2d274aba9e350674d8e7c06f [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:20.097687 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 25470
I20260926 06:24:20.097802 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:24:20.097997 4906 external_mini_cluster.cc:1428] Setting key 87598a65c01dece8bb8b4fb6b5828bba
W20260926 06:24:20.098801 25557 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 77e98fde2d274aba9e350674d8e7c06f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:24:20.098867 25557 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:24:20.098944 25545 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:24:20.099095 25545 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:24:20.100782 25545 catalog_manager.cc:1347] Generated new cluster ID: 4efc646aee4a4522b9d613f8365d9c59
I20260926 06:24:20.100852 25545 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:24:20 dist-test-slave-pd0t krb5kdc[25457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 06:24:20.108976 25559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:20.098430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55017 (local address 127.0.0.1:35789)
0926 06:24:20.098672 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:20.098681 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:20.098739 (+ 58us) server_negotiation.cc:407] Connection header received
0926 06:24:20.098826 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:20.098831 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:20.098877 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:20.099021 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:20.099867 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:20.101142 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:20.101921 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:20.102318 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:20.102459 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:20.105188 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:20.105192 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:20.105192 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:20.105203 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:20.107452 (+ 2249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:20.107838 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:20.107841 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:20.107841 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:20.107950 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:20.108167 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:20.108167 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:20.108168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:20.108390 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:20.108568 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:20.108661 (+ 93us) server_negotiation.cc:299] Negotiation successful
0926 06:24:20.108809 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":67,"threads_started":1}
I20260926 06:24:20.109882 4906 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:24:20.112155 25545 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:24:20.112629 25545 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:24:20.116225 25545 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77e98fde2d274aba9e350674d8e7c06f: Generated new TSK 0
I20260926 06:24:20.116391 25545 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:24:20.121312 25559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:20.111967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50763 (local address 127.0.0.1:35789)
0926 06:24:20.112005 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:20.112007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:20.112190 (+ 183us) server_negotiation.cc:407] Connection header received
0926 06:24:20.114724 (+ 2534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:20.114725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:20.114737 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:20.114805 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:20.115402 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:20.116165 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:20.117260 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:20.117467 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:20.117549 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:20.118853 (+ 1304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:20.118854 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:20.118855 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:20.118861 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:20.120245 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:20.120619 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:20.120620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:20.120620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:20.120670 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:20.120907 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:20.120907 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:20.120908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:20.121000 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:20.121072 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:20.121207 (+ 135us) server_negotiation.cc:299] Negotiation successful
0926 06:24:20.121260 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 06:24:20.123286 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 25470
2026-09-26T06:24:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1283 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:24:20 dist-test-slave-pd0t krb5kdc[25569](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:24:20 dist-test-slave-pd0t krb5kdc[25569](info): set up 2 sockets
Sep 26 06:24:20 dist-test-slave-pd0t krb5kdc[25569](info): commencing operation
krb5kdc: starting...
W20260926 06:24:21.162559 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:24:21 dist-test-slave-pd0t krb5kdc[25569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:24:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:24:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 06:24:21.281498 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33733
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:24:21.360448 25582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:21.360664 25582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:21.360699 25582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:21.361989 25582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:24:21.362051 25582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:21.362075 25582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:21.362107 25582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:24:21.362129 25582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:24:21.363595 25582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33733
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:21.363871 25582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:21.364086 25582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:21.368716 25588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:21.368746 25590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:21.368863 25582 server_base.cc:1034] running on GCE node
W20260926 06:24:21.368709 25587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:21.369210 25582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:21.369549 25582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:21.370713 25582 hybrid_clock.cc:648] HybridClock initialized: now 1790403861370694 us; error 33 us; skew 500 ppm
Sep 26 06:24:21 dist-test-slave-pd0t krb5kdc[25569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403861, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:21.373131 25582 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 06:24:21.373541 25582 webserver.cc:460] Webserver started at http://127.0.0.1:42857/ using document root <none> and password file <none>
I20260926 06:24:21.373780 25582 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:21.373847 25582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:21.373997 25582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:21.375083 25582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "ccd147c84fb6470f8695a42d75680986"
format_stamp: "Formatted at 2026-09-26 06:24:21 on dist-test-slave-pd0t"
server_key: "e997b8d7e23c3803c9ac5fc0fc74e4a3"
server_key_iv: "80da854d18733ad22f58e890c06101f5"
server_key_version: "encryptionkey@0"
I20260926 06:24:21.375474 25582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "ccd147c84fb6470f8695a42d75680986"
format_stamp: "Formatted at 2026-09-26 06:24:21 on dist-test-slave-pd0t"
server_key: "e997b8d7e23c3803c9ac5fc0fc74e4a3"
server_key_iv: "80da854d18733ad22f58e890c06101f5"
server_key_version: "encryptionkey@0"
I20260926 06:24:21.376711 25582 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:24:21.377415 25597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:21.377583 25582 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:24:21.377669 25582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "ccd147c84fb6470f8695a42d75680986"
format_stamp: "Formatted at 2026-09-26 06:24:21 on dist-test-slave-pd0t"
server_key: "e997b8d7e23c3803c9ac5fc0fc74e4a3"
server_key_iv: "80da854d18733ad22f58e890c06101f5"
server_key_version: "encryptionkey@0"
I20260926 06:24:21.377749 25582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:21.389869 25582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:21.391650 25582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:21.391796 25582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:21.395726 25582 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33733
I20260926 06:24:21.395750 25649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33733 every 8 connection(s)
I20260926 06:24:21.396097 25582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:24:21.397414 25650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:21.400338 25650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccd147c84fb6470f8695a42d75680986: Bootstrap starting.
I20260926 06:24:21.400843 25650 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ccd147c84fb6470f8695a42d75680986: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:21.401067 25650 log.cc:824] T 00000000000000000000000000000000 P ccd147c84fb6470f8695a42d75680986: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:21.401700 25650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccd147c84fb6470f8695a42d75680986: No bootstrap required, opened a new log
I20260926 06:24:21.403339 25650 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccd147c84fb6470f8695a42d75680986 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccd147c84fb6470f8695a42d75680986" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33733 } }
I20260926 06:24:21.403473 25650 raft_consensus.cc:374] T 00000000000000000000000000000000 P ccd147c84fb6470f8695a42d75680986 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:21.403496 25650 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccd147c84fb6470f8695a42d75680986 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccd147c84fb6470f8695a42d75680986, State: Initialized, Role: FOLLOWER
I20260926 06:24:21.403620 25650 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccd147c84fb6470f8695a42d75680986 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccd147c84fb6470f8695a42d75680986" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33733 } }
I20260926 06:24:21.403702 25650 raft_consensus.cc:388] T 00000000000000000000000000000000 P ccd147c84fb6470f8695a42d75680986 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:24:21.403725 25650 raft_consensus.cc:482] T 00000000000000000000000000000000 P ccd147c84fb6470f8695a42d75680986 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:24:21.403759 25650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccd147c84fb6470f8695a42d75680986 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:21.404247 25650 raft_consensus.cc:504] T 00000000000000000000000000000000 P ccd147c84fb6470f8695a42d75680986 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccd147c84fb6470f8695a42d75680986" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33733 } }
I20260926 06:24:21.404345 25650 leader_election.cc:302] T 00000000000000000000000000000000 P ccd147c84fb6470f8695a42d75680986 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ccd147c84fb6470f8695a42d75680986; no voters:
I20260926 06:24:21.404458 25650 leader_election.cc:288] T 00000000000000000000000000000000 P ccd147c84fb6470f8695a42d75680986 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:24:21.404527 25653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ccd147c84fb6470f8695a42d75680986 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:24:21.404670 25653 raft_consensus.cc:686] T 00000000000000000000000000000000 P ccd147c84fb6470f8695a42d75680986 [term 1 LEADER]: Becoming Leader. State: Replica: ccd147c84fb6470f8695a42d75680986, State: Running, Role: LEADER
I20260926 06:24:21.404727 25650 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccd147c84fb6470f8695a42d75680986 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:24:21.404757 25653 consensus_queue.cc:237] T 00000000000000000000000000000000 P ccd147c84fb6470f8695a42d75680986 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccd147c84fb6470f8695a42d75680986" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33733 } }
I20260926 06:24:21.405206 25655 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccd147c84fb6470f8695a42d75680986 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ccd147c84fb6470f8695a42d75680986. Latest consensus state: current_term: 1 leader_uuid: "ccd147c84fb6470f8695a42d75680986" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccd147c84fb6470f8695a42d75680986" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33733 } } }
I20260926 06:24:21.405290 25655 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccd147c84fb6470f8695a42d75680986 [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:21.405344 25654 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccd147c84fb6470f8695a42d75680986 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ccd147c84fb6470f8695a42d75680986" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccd147c84fb6470f8695a42d75680986" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33733 } } }
I20260926 06:24:21.405431 25654 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccd147c84fb6470f8695a42d75680986 [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:21.405714 25660 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:24:21.405887 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 25582
I20260926 06:24:21.406009 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:24:21.406207 4906 external_mini_cluster.cc:1428] Setting key c3bd92fdc8161229e38675ead65ece89
I20260926 06:24:21.406219 25660 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:24:21.407768 25660 catalog_manager.cc:1347] Generated new cluster ID: 9737d924740a4321a6a1166e1c018bcd
I20260926 06:24:21.407819 25660 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:24:21.412292 25672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:21.406763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38307 (local address 127.0.0.1:33733)
0926 06:24:21.406918 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:21.406926 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:21.406955 (+ 29us) server_negotiation.cc:407] Connection header received
0926 06:24:21.407064 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:21.407071 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:21.407123 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:21.407261 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:21.407589 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:21.408959 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:21.410022 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:21.410691 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:21.410801 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:21.411491 (+ 690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:21.411495 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:21.411496 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:21.411705 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:21.411742 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:21.411756 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:21.411829 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:21.411910 (+ 81us) server_negotiation.cc:299] Negotiation successful
0926 06:24:21.412084 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":64,"threads_started":1}
I20260926 06:24:21.413192 4906 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:24:21.418016 25672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:21.414683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58687 (local address 127.0.0.1:33733)
0926 06:24:21.414722 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:21.414724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:21.414865 (+ 141us) server_negotiation.cc:407] Connection header received
0926 06:24:21.414911 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:21.414912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:21.414927 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:21.414968 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:21.415103 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:21.415790 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:21.416797 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:21.417090 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:21.417149 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:21.417785 (+ 636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:21.417786 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:21.417786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:21.417791 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:21.417805 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:21.417808 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:21.417866 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:21.417922 (+ 56us) server_negotiation.cc:299] Negotiation successful
0926 06:24:21.417949 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:24:21.429564 25660 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:24:21.430054 25660 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:24:21.439231 25660 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ccd147c84fb6470f8695a42d75680986: Generated new TSK 0
I20260926 06:24:21.439409 25660 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:24:21.442039 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 25582
2026-09-26T06:24:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1317 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:24:21 dist-test-slave-pd0t krb5kdc[25681](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:24:21 dist-test-slave-pd0t krb5kdc[25681](info): set up 2 sockets
Sep 26 06:24:21 dist-test-slave-pd0t krb5kdc[25681](info): commencing operation
krb5kdc: starting...
W20260926 06:24:22.462555 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:24:22 dist-test-slave-pd0t krb5kdc[25681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:24:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:24:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 06:24:22.576565 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38499
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38499
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:24:22.655463 25694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:22.655648 25694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:22.655670 25694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:22.656775 25694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:24:22.656817 25694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:22.656832 25694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:22.656874 25694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:24:22.656910 25694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:24:22.658354 25694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38499
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:22.658659 25694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:22.658883 25694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:22.663298 25699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:22.663360 25700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:22.663360 25702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:22.663551 25694 server_base.cc:1034] running on GCE node
I20260926 06:24:22.663744 25694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:22.663954 25694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:22.665092 25694 hybrid_clock.cc:648] HybridClock initialized: now 1790403862665081 us; error 25 us; skew 500 ppm
Sep 26 06:24:22 dist-test-slave-pd0t krb5kdc[25681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403862, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:22.673565 25694 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 06:24:22.673926 25694 webserver.cc:460] Webserver started at http://127.0.0.1:41141/ using document root <none> and password file <none>
I20260926 06:24:22.674163 25694 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:22.674232 25694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:22.674386 25694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:22.675428 25694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "f3f2b58752904ef3a4124b50895c8d5c"
format_stamp: "Formatted at 2026-09-26 06:24:22 on dist-test-slave-pd0t"
server_key: "4e6a8e0a71af74eb160bf2c7f0397b5c"
server_key_iv: "0eddcedc6dc23e7ca2593ca181d5e6dc"
server_key_version: "encryptionkey@0"
I20260926 06:24:22.675781 25694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "f3f2b58752904ef3a4124b50895c8d5c"
format_stamp: "Formatted at 2026-09-26 06:24:22 on dist-test-slave-pd0t"
server_key: "4e6a8e0a71af74eb160bf2c7f0397b5c"
server_key_iv: "0eddcedc6dc23e7ca2593ca181d5e6dc"
server_key_version: "encryptionkey@0"
I20260926 06:24:22.677089 25694 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:24:22.677819 25709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:22.678007 25694 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:24:22.678086 25694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "f3f2b58752904ef3a4124b50895c8d5c"
format_stamp: "Formatted at 2026-09-26 06:24:22 on dist-test-slave-pd0t"
server_key: "4e6a8e0a71af74eb160bf2c7f0397b5c"
server_key_iv: "0eddcedc6dc23e7ca2593ca181d5e6dc"
server_key_version: "encryptionkey@0"
I20260926 06:24:22.678136 25694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:22.681308 25694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:22.683061 25694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:22.683180 25694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:22.686728 25694 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38499
I20260926 06:24:22.686806 25761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38499 every 8 connection(s)
I20260926 06:24:22.687110 25694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:24:22.687988 25762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:22.690048 25762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3f2b58752904ef3a4124b50895c8d5c: Bootstrap starting.
I20260926 06:24:22.690565 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 25694
I20260926 06:24:22.690572 25762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3f2b58752904ef3a4124b50895c8d5c: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:22.690723 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:24:22.690927 4906 external_mini_cluster.cc:1428] Setting key 6440a4205b855ec13c21d8edda135176
I20260926 06:24:22.690968 25762 log.cc:824] T 00000000000000000000000000000000 P f3f2b58752904ef3a4124b50895c8d5c: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:22.691723 25762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3f2b58752904ef3a4124b50895c8d5c: No bootstrap required, opened a new log
I20260926 06:24:22.693377 25766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:22.691366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52081 (local address 127.0.0.1:38499)
0926 06:24:22.691520 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:22.691528 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:22.691562 (+ 34us) server_negotiation.cc:407] Connection header received
0926 06:24:22.692167 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:22.692175 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:22.692208 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:22.692349 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:22.692966 (+ 617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:22.692969 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:22.692970 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:22.692974 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:22.693017 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:22.693038 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:22.693126 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:22.693158 (+ 32us) server_negotiation.cc:299] Negotiation successful
0926 06:24:22.693196 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":72,"threads_started":1}
I20260926 06:24:22.694305 25762 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3f2b58752904ef3a4124b50895c8d5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3f2b58752904ef3a4124b50895c8d5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38499 } }
I20260926 06:24:22.694485 25762 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3f2b58752904ef3a4124b50895c8d5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:22.694525 25762 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3f2b58752904ef3a4124b50895c8d5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3f2b58752904ef3a4124b50895c8d5c, State: Initialized, Role: FOLLOWER
I20260926 06:24:22.694566 4906 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:24:22.694674 25762 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3f2b58752904ef3a4124b50895c8d5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3f2b58752904ef3a4124b50895c8d5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38499 } }
I20260926 06:24:22.694766 25762 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3f2b58752904ef3a4124b50895c8d5c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:24:22.694794 25762 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3f2b58752904ef3a4124b50895c8d5c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:24:22.694829 25762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3f2b58752904ef3a4124b50895c8d5c [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:22.695513 25762 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3f2b58752904ef3a4124b50895c8d5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3f2b58752904ef3a4124b50895c8d5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38499 } }
I20260926 06:24:22.695628 25762 leader_election.cc:302] T 00000000000000000000000000000000 P f3f2b58752904ef3a4124b50895c8d5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3f2b58752904ef3a4124b50895c8d5c; no voters:
I20260926 06:24:22.695833 25762 leader_election.cc:288] T 00000000000000000000000000000000 P f3f2b58752904ef3a4124b50895c8d5c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:24:22.695892 25770 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3f2b58752904ef3a4124b50895c8d5c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:24:22.696089 25770 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3f2b58752904ef3a4124b50895c8d5c [term 1 LEADER]: Becoming Leader. State: Replica: f3f2b58752904ef3a4124b50895c8d5c, State: Running, Role: LEADER
I20260926 06:24:22.696103 25762 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3f2b58752904ef3a4124b50895c8d5c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:24:22.696223 25770 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3f2b58752904ef3a4124b50895c8d5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3f2b58752904ef3a4124b50895c8d5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38499 } }
I20260926 06:24:22.696575 25770 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3f2b58752904ef3a4124b50895c8d5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3f2b58752904ef3a4124b50895c8d5c. Latest consensus state: current_term: 1 leader_uuid: "f3f2b58752904ef3a4124b50895c8d5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3f2b58752904ef3a4124b50895c8d5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38499 } } }
I20260926 06:24:22.696659 25770 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3f2b58752904ef3a4124b50895c8d5c [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:22.696796 25772 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3f2b58752904ef3a4124b50895c8d5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3f2b58752904ef3a4124b50895c8d5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3f2b58752904ef3a4124b50895c8d5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38499 } } }
I20260926 06:24:22.696882 25781 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:24:22.697161 25772 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3f2b58752904ef3a4124b50895c8d5c [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:22.697561 25781 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:24:22.698489 25766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:22.696182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43797 (local address 127.0.0.1:38499)
0926 06:24:22.696214 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:22.696216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:22.697346 (+ 1130us) server_negotiation.cc:407] Connection header received
0926 06:24:22.697468 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:22.697469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:22.697475 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:22.697530 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:22.698297 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:22.698298 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:22.698299 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:22.698302 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:22.698313 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:22.698317 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:22.698364 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:22.698442 (+ 78us) server_negotiation.cc:299] Negotiation successful
0926 06:24:22.698452 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 06:24:22.699601 25781 catalog_manager.cc:1347] Generated new cluster ID: 63d792b6044e46a4be68d267fb9f67a1
I20260926 06:24:22.699658 25781 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:24:22.723819 25781 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:24:22.723959 25781 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:24:22.726832 25781 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3f2b58752904ef3a4124b50895c8d5c: Generated new TSK 0
I20260926 06:24:22.727006 25781 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:24:22.736114 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 25694
2026-09-26T06:24:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1294 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:24:22 dist-test-slave-pd0t krb5kdc[25793](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:24:22 dist-test-slave-pd0t krb5kdc[25793](info): set up 2 sockets
Sep 26 06:24:22 dist-test-slave-pd0t krb5kdc[25793](info): commencing operation
krb5kdc: starting...
W20260926 06:24:23.754529 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.001s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:24:23 dist-test-slave-pd0t krb5kdc[25793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:24:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:24:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 06:24:23.865672 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40883
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:24:23.941368 25806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:23.941613 25806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:23.941635 25806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:23.942824 25806 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:24:23.942871 25806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:23.942885 25806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:23.942945 25806 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:24:23.942966 25806 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:24:23.944476 25806 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40883
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:23.944742 25806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:23.944972 25806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:23.949676 25814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:23.949688 25812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:23.949712 25811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:23.949999 25806 server_base.cc:1034] running on GCE node
I20260926 06:24:23.950222 25806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:23.950490 25806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:23.951642 25806 hybrid_clock.cc:648] HybridClock initialized: now 1790403863951618 us; error 37 us; skew 500 ppm
Sep 26 06:24:23 dist-test-slave-pd0t krb5kdc[25793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403863, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:23.954113 25806 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 06:24:23.954560 25806 webserver.cc:460] Webserver started at http://127.0.0.1:41641/ using document root <none> and password file <none>
I20260926 06:24:23.954787 25806 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:23.954833 25806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:23.954918 25806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:23.955847 25806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "56e9d7a9256946b8acc9870990b721b5"
format_stamp: "Formatted at 2026-09-26 06:24:23 on dist-test-slave-pd0t"
server_key: "e801d275b6a8e4dd58e7d097a270d31a"
server_key_iv: "88c5a2b0d4d6d6b671ab63657ed27edc"
server_key_version: "encryptionkey@0"
I20260926 06:24:23.956138 25806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "56e9d7a9256946b8acc9870990b721b5"
format_stamp: "Formatted at 2026-09-26 06:24:23 on dist-test-slave-pd0t"
server_key: "e801d275b6a8e4dd58e7d097a270d31a"
server_key_iv: "88c5a2b0d4d6d6b671ab63657ed27edc"
server_key_version: "encryptionkey@0"
I20260926 06:24:23.957378 25806 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:24:23.958057 25821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:23.958243 25806 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:24:23.958312 25806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "56e9d7a9256946b8acc9870990b721b5"
format_stamp: "Formatted at 2026-09-26 06:24:23 on dist-test-slave-pd0t"
server_key: "e801d275b6a8e4dd58e7d097a270d31a"
server_key_iv: "88c5a2b0d4d6d6b671ab63657ed27edc"
server_key_version: "encryptionkey@0"
I20260926 06:24:23.958360 25806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:23.970429 25806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:23.972142 25806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:23.972267 25806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:23.976187 25806 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40883
I20260926 06:24:23.976248 25873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40883 every 8 connection(s)
I20260926 06:24:23.976567 25806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:24:23.977499 25874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:23.979351 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 25806
I20260926 06:24:23.979458 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:24:23.979619 4906 external_mini_cluster.cc:1428] Setting key c22bf85f9c82cef772cdfabd885af930
I20260926 06:24:23.979868 25874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56e9d7a9256946b8acc9870990b721b5: Bootstrap starting.
I20260926 06:24:23.980583 25874 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56e9d7a9256946b8acc9870990b721b5: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:23.980872 25874 log.cc:824] T 00000000000000000000000000000000 P 56e9d7a9256946b8acc9870990b721b5: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:23.981683 25874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56e9d7a9256946b8acc9870990b721b5: No bootstrap required, opened a new log
I20260926 06:24:23.983798 25874 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56e9d7a9256946b8acc9870990b721b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56e9d7a9256946b8acc9870990b721b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40883 } }
I20260926 06:24:23.983960 25874 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56e9d7a9256946b8acc9870990b721b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:23.984031 25874 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56e9d7a9256946b8acc9870990b721b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56e9d7a9256946b8acc9870990b721b5, State: Initialized, Role: FOLLOWER
I20260926 06:24:23.984174 25874 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56e9d7a9256946b8acc9870990b721b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56e9d7a9256946b8acc9870990b721b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40883 } }
I20260926 06:24:23.984236 25874 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56e9d7a9256946b8acc9870990b721b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:24:23.984256 25874 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56e9d7a9256946b8acc9870990b721b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:24:23.984315 25874 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56e9d7a9256946b8acc9870990b721b5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:23.984973 25874 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56e9d7a9256946b8acc9870990b721b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56e9d7a9256946b8acc9870990b721b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40883 } }
I20260926 06:24:23.985093 25874 leader_election.cc:302] T 00000000000000000000000000000000 P 56e9d7a9256946b8acc9870990b721b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56e9d7a9256946b8acc9870990b721b5; no voters:
I20260926 06:24:23.985266 25874 leader_election.cc:288] T 00000000000000000000000000000000 P 56e9d7a9256946b8acc9870990b721b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:24:23.985291 25879 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56e9d7a9256946b8acc9870990b721b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:24:23.985502 25879 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56e9d7a9256946b8acc9870990b721b5 [term 1 LEADER]: Becoming Leader. State: Replica: 56e9d7a9256946b8acc9870990b721b5, State: Running, Role: LEADER
I20260926 06:24:23.985522 25874 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56e9d7a9256946b8acc9870990b721b5 [sys.catalog]: configured and running, proceeding with master startup.
Sep 26 06:24:23 dist-test-slave-pd0t krb5kdc[25793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 06:24:23.986922 25879 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56e9d7a9256946b8acc9870990b721b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56e9d7a9256946b8acc9870990b721b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40883 } }
I20260926 06:24:23.987484 25880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56e9d7a9256946b8acc9870990b721b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56e9d7a9256946b8acc9870990b721b5. Latest consensus state: current_term: 1 leader_uuid: "56e9d7a9256946b8acc9870990b721b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56e9d7a9256946b8acc9870990b721b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40883 } } }
I20260926 06:24:23.988078 25880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56e9d7a9256946b8acc9870990b721b5 [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:23.987648 25881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56e9d7a9256946b8acc9870990b721b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56e9d7a9256946b8acc9870990b721b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56e9d7a9256946b8acc9870990b721b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40883 } } }
I20260926 06:24:23.988241 25881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56e9d7a9256946b8acc9870990b721b5 [sys.catalog]: This master's current role is: LEADER
W20260926 06:24:23.988476 25893 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 56e9d7a9256946b8acc9870990b721b5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:24:23.988521 25893 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:24:23.988597 25896 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:24:23.988762 25896 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:24:23.990207 25896 catalog_manager.cc:1347] Generated new cluster ID: 5ba5e3bdf14a4bdf8716eeb508930e28
I20260926 06:24:23.990278 25896 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:24:23.991333 25877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:23.980077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:33969 (local address 127.0.0.1:40883)
0926 06:24:23.980257 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:23.980266 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:23.980301 (+ 35us) server_negotiation.cc:407] Connection header received
0926 06:24:23.980760 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:23.980768 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:23.980831 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:23.980966 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:23.981756 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:23.982851 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:23.983982 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:23.984541 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:23.984617 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:23.987502 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:23.987506 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:23.987507 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:23.987517 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:23.989389 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:23.989814 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:23.989817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:23.989818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:23.989934 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:23.990201 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:23.990202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:23.990202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:23.990500 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:23.990966 (+ 466us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:23.991049 (+ 83us) server_negotiation.cc:299] Negotiation successful
0926 06:24:23.991135 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":93,"threads_started":1}
I20260926 06:24:23.992260 4906 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:24:24.000536 25877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:23.993591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:39335 (local address 127.0.0.1:40883)
0926 06:24:23.993618 (+ 27us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:23.993620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:23.993752 (+ 132us) server_negotiation.cc:407] Connection header received
0926 06:24:23.993797 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:23.993798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:23.993806 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:23.993857 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:23.994439 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:23.995175 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:23.996142 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:23.996456 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:23.996512 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:23.997830 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:23.997831 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:23.997831 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:23.997837 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:23.999335 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:23.999761 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:23.999762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:23.999762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:23.999814 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:24.000040 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:24.000040 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:24.000041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:24.000149 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:24.000233 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:24.000399 (+ 166us) server_negotiation.cc:299] Negotiation successful
0926 06:24:24.000457 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260926 06:24:24.006803 25896 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:24:24.007370 25896 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:24:24.014636 25896 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56e9d7a9256946b8acc9870990b721b5: Generated new TSK 0
I20260926 06:24:24.014818 25896 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:24:24.017047 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 25806
W20260926 06:24:24.020982 25791 connection.cc:535] client connection to 127.0.0.1:40883 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40883): Connection reset by peer (error 104)
2026-09-26T06:24:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1280 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:24:24 dist-test-slave-pd0t krb5kdc[25905](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:24:24 dist-test-slave-pd0t krb5kdc[25905](info): set up 2 sockets
Sep 26 06:24:24 dist-test-slave-pd0t krb5kdc[25905](info): commencing operation
krb5kdc: starting...
W20260926 06:24:25.042662 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:24:25 dist-test-slave-pd0t krb5kdc[25905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:24:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:24:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 06:24:25.153431 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43521
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:24:25.231904 25918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:25.232129 25918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:25.232168 25918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:25.233294 25918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:24:25.233366 25918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:25.233390 25918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:25.233422 25918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:24:25.233448 25918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:24:25.234908 25918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:25.235155 25918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:25.235366 25918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:25.239986 25923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:25.239981 25924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:25.239977 25926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:25.240079 25918 server_base.cc:1034] running on GCE node
I20260926 06:24:25.240450 25918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:25.240743 25918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:25.241938 25918 hybrid_clock.cc:648] HybridClock initialized: now 1790403865241926 us; error 28 us; skew 500 ppm
Sep 26 06:24:25 dist-test-slave-pd0t krb5kdc[25905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403865, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:25.244277 25918 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 06:24:25.244704 25918 webserver.cc:460] Webserver started at http://127.0.0.1:42211/ using document root <none> and password file <none>
I20260926 06:24:25.244935 25918 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:25.245000 25918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:25.245128 25918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:25.246129 25918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "c444788ba6674759aa3b11a711841d9d"
format_stamp: "Formatted at 2026-09-26 06:24:25 on dist-test-slave-pd0t"
server_key: "a61b197e372317af508f210499be93e7"
server_key_iv: "294bcf12c228828df39d69821f1361e0"
server_key_version: "encryptionkey@0"
I20260926 06:24:25.246471 25918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "c444788ba6674759aa3b11a711841d9d"
format_stamp: "Formatted at 2026-09-26 06:24:25 on dist-test-slave-pd0t"
server_key: "a61b197e372317af508f210499be93e7"
server_key_iv: "294bcf12c228828df39d69821f1361e0"
server_key_version: "encryptionkey@0"
I20260926 06:24:25.247777 25918 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:24:25.248581 25933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:25.248754 25918 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:24:25.248847 25918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "c444788ba6674759aa3b11a711841d9d"
format_stamp: "Formatted at 2026-09-26 06:24:25 on dist-test-slave-pd0t"
server_key: "a61b197e372317af508f210499be93e7"
server_key_iv: "294bcf12c228828df39d69821f1361e0"
server_key_version: "encryptionkey@0"
I20260926 06:24:25.248935 25918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:25.256459 25918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:25.258234 25918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:25.258406 25918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:25.262290 25918 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43521
I20260926 06:24:25.262321 25985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43521 every 8 connection(s)
I20260926 06:24:25.262723 25918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:24:25.263693 25986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:25.265945 25986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c444788ba6674759aa3b11a711841d9d: Bootstrap starting.
I20260926 06:24:25.266508 25986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c444788ba6674759aa3b11a711841d9d: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:25.266829 25986 log.cc:824] T 00000000000000000000000000000000 P c444788ba6674759aa3b11a711841d9d: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:25.267488 25986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c444788ba6674759aa3b11a711841d9d: No bootstrap required, opened a new log
I20260926 06:24:25.267493 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 25918
I20260926 06:24:25.267634 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:24:25.267830 4906 external_mini_cluster.cc:1428] Setting key 8c3133541d093d857aa50b2eb394b9cd
I20260926 06:24:25.269433 25986 raft_consensus.cc:348] T 00000000000000000000000000000000 P c444788ba6674759aa3b11a711841d9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c444788ba6674759aa3b11a711841d9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43521 } }
I20260926 06:24:25.269582 25986 raft_consensus.cc:374] T 00000000000000000000000000000000 P c444788ba6674759aa3b11a711841d9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:25.269644 25986 raft_consensus.cc:729] T 00000000000000000000000000000000 P c444788ba6674759aa3b11a711841d9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c444788ba6674759aa3b11a711841d9d, State: Initialized, Role: FOLLOWER
I20260926 06:24:25.269778 25986 consensus_queue.cc:260] T 00000000000000000000000000000000 P c444788ba6674759aa3b11a711841d9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c444788ba6674759aa3b11a711841d9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43521 } }
I20260926 06:24:25.269873 25986 raft_consensus.cc:388] T 00000000000000000000000000000000 P c444788ba6674759aa3b11a711841d9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:24:25.269922 25986 raft_consensus.cc:482] T 00000000000000000000000000000000 P c444788ba6674759aa3b11a711841d9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:24:25.269970 25986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c444788ba6674759aa3b11a711841d9d [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:25.270478 25986 raft_consensus.cc:504] T 00000000000000000000000000000000 P c444788ba6674759aa3b11a711841d9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c444788ba6674759aa3b11a711841d9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43521 } }
I20260926 06:24:25.270591 25986 leader_election.cc:302] T 00000000000000000000000000000000 P c444788ba6674759aa3b11a711841d9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c444788ba6674759aa3b11a711841d9d; no voters:
I20260926 06:24:25.270756 25986 leader_election.cc:288] T 00000000000000000000000000000000 P c444788ba6674759aa3b11a711841d9d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:24:25.270788 25991 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c444788ba6674759aa3b11a711841d9d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:24:25.270917 25991 raft_consensus.cc:686] T 00000000000000000000000000000000 P c444788ba6674759aa3b11a711841d9d [term 1 LEADER]: Becoming Leader. State: Replica: c444788ba6674759aa3b11a711841d9d, State: Running, Role: LEADER
I20260926 06:24:25.271013 25986 sys_catalog.cc:564] T 00000000000000000000000000000000 P c444788ba6674759aa3b11a711841d9d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:24:25.271047 25991 consensus_queue.cc:237] T 00000000000000000000000000000000 P c444788ba6674759aa3b11a711841d9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c444788ba6674759aa3b11a711841d9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43521 } }
I20260926 06:24:25.271404 25991 sys_catalog.cc:455] T 00000000000000000000000000000000 P c444788ba6674759aa3b11a711841d9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c444788ba6674759aa3b11a711841d9d. Latest consensus state: current_term: 1 leader_uuid: "c444788ba6674759aa3b11a711841d9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c444788ba6674759aa3b11a711841d9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43521 } } }
I20260926 06:24:25.271617 25991 sys_catalog.cc:458] T 00000000000000000000000000000000 P c444788ba6674759aa3b11a711841d9d [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:25.271554 25992 sys_catalog.cc:455] T 00000000000000000000000000000000 P c444788ba6674759aa3b11a711841d9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c444788ba6674759aa3b11a711841d9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c444788ba6674759aa3b11a711841d9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43521 } } }
I20260926 06:24:25.272022 25992 sys_catalog.cc:458] T 00000000000000000000000000000000 P c444788ba6674759aa3b11a711841d9d [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:25.272194 26000 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:24:25.272650 26000 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:24:25.274178 26000 catalog_manager.cc:1347] Generated new cluster ID: a0f8adfca295406b82a1b35aff71fa48
I20260926 06:24:25.274233 26000 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:24:25.274824 25990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:25.268395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:35521 (local address 127.0.0.1:43521)
0926 06:24:25.268569 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:25.268576 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:25.268599 (+ 23us) server_negotiation.cc:407] Connection header received
0926 06:24:25.269418 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:25.269424 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:25.269468 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:25.269608 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:25.269812 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:25.271382 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:25.272534 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:25.272993 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:25.273074 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:25.274014 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:25.274017 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:25.274017 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:25.274242 (+ 225us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:25.274279 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:25.274293 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:25.274383 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:25.274542 (+ 159us) server_negotiation.cc:299] Negotiation successful
0926 06:24:25.274565 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":66,"threads_started":1}
I20260926 06:24:25.275643 4906 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:24:25.280321 26000 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:24:25.280546 25990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:25.276954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:46889 (local address 127.0.0.1:43521)
0926 06:24:25.276995 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:25.276997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:25.277143 (+ 146us) server_negotiation.cc:407] Connection header received
0926 06:24:25.277218 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:25.277218 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:25.277227 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:25.277252 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:25.277424 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:25.278107 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:25.279085 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:25.279436 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:25.279474 (+ 38us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:25.280261 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:25.280262 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:25.280262 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:25.280266 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:25.280277 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:25.280281 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:25.280346 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:25.280468 (+ 122us) server_negotiation.cc:299] Negotiation successful
0926 06:24:25.280479 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 06:24:25.281023 26000 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:24:25.287184 26000 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c444788ba6674759aa3b11a711841d9d: Generated new TSK 0
I20260926 06:24:25.287372 26000 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:24:25.292220 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 25918
W20260926 06:24:25.296451 25903 connection.cc:535] client connection to 127.0.0.1:43521 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43521): Connection reset by peer (error 104)
2026-09-26T06:24:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1275 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:24:25 dist-test-slave-pd0t krb5kdc[26017](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:24:25 dist-test-slave-pd0t krb5kdc[26017](info): set up 2 sockets
Sep 26 06:24:25 dist-test-slave-pd0t krb5kdc[26017](info): commencing operation
krb5kdc: starting...
W20260926 06:24:26.314658 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:24:26 dist-test-slave-pd0t krb5kdc[26017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:24:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:24:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 06:24:26.430099 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44101
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:24:26.508268 26030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:26.508466 26030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:26.508488 26030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:26.509559 26030 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:24:26.509591 26030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:26.509606 26030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:26.509618 26030 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:24:26.509630 26030 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:24:26.511059 26030 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:26.511240 26030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:26.511406 26030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:26.516214 26038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:26.516266 26035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:26.516214 26036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:26.516443 26030 server_base.cc:1034] running on GCE node
I20260926 06:24:26.516724 26030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:26.517062 26030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:26.518231 26030 hybrid_clock.cc:648] HybridClock initialized: now 1790403866518210 us; error 37 us; skew 500 ppm
Sep 26 06:24:26 dist-test-slave-pd0t krb5kdc[26017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403866, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:26.520673 26030 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 06:24:26.521178 26030 webserver.cc:460] Webserver started at http://127.0.0.1:46027/ using document root <none> and password file <none>
I20260926 06:24:26.521412 26030 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:26.521488 26030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:26.521629 26030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:26.522681 26030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "69dedf9d24a94af9829cb398c6b211de"
format_stamp: "Formatted at 2026-09-26 06:24:26 on dist-test-slave-pd0t"
server_key: "b899ff13e043ccf33ab309f29760988a"
server_key_iv: "e7753143958df56868f52b977a241cee"
server_key_version: "encryptionkey@0"
I20260926 06:24:26.523031 26030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "69dedf9d24a94af9829cb398c6b211de"
format_stamp: "Formatted at 2026-09-26 06:24:26 on dist-test-slave-pd0t"
server_key: "b899ff13e043ccf33ab309f29760988a"
server_key_iv: "e7753143958df56868f52b977a241cee"
server_key_version: "encryptionkey@0"
I20260926 06:24:26.524288 26030 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:24:26.524967 26045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:26.525138 26030 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:24:26.525236 26030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "69dedf9d24a94af9829cb398c6b211de"
format_stamp: "Formatted at 2026-09-26 06:24:26 on dist-test-slave-pd0t"
server_key: "b899ff13e043ccf33ab309f29760988a"
server_key_iv: "e7753143958df56868f52b977a241cee"
server_key_version: "encryptionkey@0"
I20260926 06:24:26.525313 26030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:26.528592 26030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:26.530359 26030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:26.530520 26030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:26.534492 26030 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44101
I20260926 06:24:26.534519 26097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44101 every 8 connection(s)
I20260926 06:24:26.534847 26030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:24:26.535755 26098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:26.537794 26098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69dedf9d24a94af9829cb398c6b211de: Bootstrap starting.
I20260926 06:24:26.538286 26098 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69dedf9d24a94af9829cb398c6b211de: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:26.538509 26098 log.cc:824] T 00000000000000000000000000000000 P 69dedf9d24a94af9829cb398c6b211de: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:26.539191 26098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69dedf9d24a94af9829cb398c6b211de: No bootstrap required, opened a new log
I20260926 06:24:26.540633 26098 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69dedf9d24a94af9829cb398c6b211de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69dedf9d24a94af9829cb398c6b211de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44101 } }
I20260926 06:24:26.540755 26098 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69dedf9d24a94af9829cb398c6b211de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:26.540777 26098 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69dedf9d24a94af9829cb398c6b211de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69dedf9d24a94af9829cb398c6b211de, State: Initialized, Role: FOLLOWER
I20260926 06:24:26.540866 26098 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69dedf9d24a94af9829cb398c6b211de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69dedf9d24a94af9829cb398c6b211de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44101 } }
I20260926 06:24:26.540937 26098 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69dedf9d24a94af9829cb398c6b211de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:24:26.540958 26098 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69dedf9d24a94af9829cb398c6b211de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:24:26.540983 26098 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69dedf9d24a94af9829cb398c6b211de [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:26.541412 26098 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69dedf9d24a94af9829cb398c6b211de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69dedf9d24a94af9829cb398c6b211de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44101 } }
I20260926 06:24:26.541487 26098 leader_election.cc:302] T 00000000000000000000000000000000 P 69dedf9d24a94af9829cb398c6b211de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69dedf9d24a94af9829cb398c6b211de; no voters:
I20260926 06:24:26.541596 26098 leader_election.cc:288] T 00000000000000000000000000000000 P 69dedf9d24a94af9829cb398c6b211de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:24:26.541672 26101 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69dedf9d24a94af9829cb398c6b211de [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:24:26.541832 26101 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69dedf9d24a94af9829cb398c6b211de [term 1 LEADER]: Becoming Leader. State: Replica: 69dedf9d24a94af9829cb398c6b211de, State: Running, Role: LEADER
I20260926 06:24:26.541848 26098 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69dedf9d24a94af9829cb398c6b211de [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:24:26.541916 26101 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69dedf9d24a94af9829cb398c6b211de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69dedf9d24a94af9829cb398c6b211de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44101 } }
I20260926 06:24:26.542261 26102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69dedf9d24a94af9829cb398c6b211de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69dedf9d24a94af9829cb398c6b211de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69dedf9d24a94af9829cb398c6b211de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44101 } } }
I20260926 06:24:26.542384 26102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69dedf9d24a94af9829cb398c6b211de [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:26.542269 26103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69dedf9d24a94af9829cb398c6b211de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69dedf9d24a94af9829cb398c6b211de. Latest consensus state: current_term: 1 leader_uuid: "69dedf9d24a94af9829cb398c6b211de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69dedf9d24a94af9829cb398c6b211de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44101 } } }
I20260926 06:24:26.542611 26103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69dedf9d24a94af9829cb398c6b211de [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:26.542611 26105 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:24:26.543280 26105 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:24:26.544178 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 26030
I20260926 06:24:26.544301 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:24:26.544476 4906 external_mini_cluster.cc:1428] Setting key 92b3d539ca69e6d9109923d8bd4ab2a0
I20260926 06:24:26.545248 26105 catalog_manager.cc:1347] Generated new cluster ID: 5e5e2c0b5ea84bc297dab758e1cb8a5e
I20260926 06:24:26.545313 26105 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:24:26.547287 26121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:26.544913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:49319 (local address 127.0.0.1:44101)
0926 06:24:26.545097 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:26.545104 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:26.545489 (+ 385us) server_negotiation.cc:407] Connection header received
0926 06:24:26.545609 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:26.545616 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:26.545645 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:26.545771 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:26.546783 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:26.546786 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:26.546786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:26.546791 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:26.546837 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:26.546850 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:26.546905 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:26.547076 (+ 171us) server_negotiation.cc:299] Negotiation successful
0926 06:24:26.547118 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":109,"threads_started":1}
I20260926 06:24:26.548153 4906 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:24:26.550762 26121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:26.549557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:59591 (local address 127.0.0.1:44101)
0926 06:24:26.549587 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:26.549589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:26.549812 (+ 223us) server_negotiation.cc:407] Connection header received
0926 06:24:26.549893 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:26.549894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:26.549899 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:26.549924 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:26.550477 (+ 553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:26.550478 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:26.550479 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:26.550482 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:26.550493 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:26.550497 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:26.550522 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:26.550638 (+ 116us) server_negotiation.cc:299] Negotiation successful
0926 06:24:26.550651 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 06:24:26.561802 26105 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:24:26.562041 26105 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:24:26.566213 26105 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69dedf9d24a94af9829cb398c6b211de: Generated new TSK 0
I20260926 06:24:26.566362 26105 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:24:26.570779 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 26030
2026-09-26T06:24:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1279 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:24:26 dist-test-slave-pd0t krb5kdc[26129](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:24:26 dist-test-slave-pd0t krb5kdc[26129](info): set up 2 sockets
Sep 26 06:24:26 dist-test-slave-pd0t krb5kdc[26129](info): commencing operation
krb5kdc: starting...
W20260926 06:24:27.594743 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:24:27 dist-test-slave-pd0t krb5kdc[26129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:24:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:24:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 06:24:27.705339 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39295
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39295
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:24:27.783386 26142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:27.783601 26142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:27.783639 26142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:27.784757 26142 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:24:27.784822 26142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:27.784845 26142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:27.784865 26142 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:24:27.784893 26142 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:24:27.786340 26142 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39295
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:27.786634 26142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:27.786851 26142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:27.791472 26147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:27.791486 26148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:27.791945 26150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:27.791985 26142 server_base.cc:1034] running on GCE node
I20260926 06:24:27.792258 26142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:27.792532 26142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:27.793695 26142 hybrid_clock.cc:648] HybridClock initialized: now 1790403867793676 us; error 52 us; skew 500 ppm
Sep 26 06:24:27 dist-test-slave-pd0t krb5kdc[26129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403867, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:27.801710 26142 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 06:24:27.802153 26142 webserver.cc:460] Webserver started at http://127.0.0.1:39195/ using document root <none> and password file <none>
I20260926 06:24:27.802387 26142 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:27.802462 26142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:27.802652 26142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:27.803655 26142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "d8cd7ba7344046118603015f10dc624f"
format_stamp: "Formatted at 2026-09-26 06:24:27 on dist-test-slave-pd0t"
server_key: "bf57c9288cacfea7b61f17ebd971180e"
server_key_iv: "941e5586c663e6c4e1738bf98b932f46"
server_key_version: "encryptionkey@0"
I20260926 06:24:27.804008 26142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "d8cd7ba7344046118603015f10dc624f"
format_stamp: "Formatted at 2026-09-26 06:24:27 on dist-test-slave-pd0t"
server_key: "bf57c9288cacfea7b61f17ebd971180e"
server_key_iv: "941e5586c663e6c4e1738bf98b932f46"
server_key_version: "encryptionkey@0"
I20260926 06:24:27.805244 26142 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:24:27.805944 26157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:27.806107 26142 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:24:27.806205 26142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "d8cd7ba7344046118603015f10dc624f"
format_stamp: "Formatted at 2026-09-26 06:24:27 on dist-test-slave-pd0t"
server_key: "bf57c9288cacfea7b61f17ebd971180e"
server_key_iv: "941e5586c663e6c4e1738bf98b932f46"
server_key_version: "encryptionkey@0"
I20260926 06:24:27.806288 26142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:27.819027 26142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:27.820689 26142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:27.820844 26142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:27.824795 26142 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39295
I20260926 06:24:27.824827 26209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39295 every 8 connection(s)
I20260926 06:24:27.825182 26142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:24:27.826130 26210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:27.828243 26210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8cd7ba7344046118603015f10dc624f: Bootstrap starting.
I20260926 06:24:27.828804 26210 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8cd7ba7344046118603015f10dc624f: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:27.829087 26210 log.cc:824] T 00000000000000000000000000000000 P d8cd7ba7344046118603015f10dc624f: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:27.829586 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 26142
I20260926 06:24:27.829720 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:24:27.829910 4906 external_mini_cluster.cc:1428] Setting key 957de302a686d48d9c353dc1f35b3224
I20260926 06:24:27.829922 26210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8cd7ba7344046118603015f10dc624f: No bootstrap required, opened a new log
I20260926 06:24:27.831746 26210 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8cd7ba7344046118603015f10dc624f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8cd7ba7344046118603015f10dc624f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39295 } }
I20260926 06:24:27.831923 26210 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8cd7ba7344046118603015f10dc624f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:27.831964 26210 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8cd7ba7344046118603015f10dc624f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8cd7ba7344046118603015f10dc624f, State: Initialized, Role: FOLLOWER
I20260926 06:24:27.832149 26210 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8cd7ba7344046118603015f10dc624f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8cd7ba7344046118603015f10dc624f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39295 } }
I20260926 06:24:27.832244 26210 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8cd7ba7344046118603015f10dc624f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:24:27.832279 26210 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8cd7ba7344046118603015f10dc624f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:24:27.832340 26210 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8cd7ba7344046118603015f10dc624f [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:27.832985 26210 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8cd7ba7344046118603015f10dc624f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8cd7ba7344046118603015f10dc624f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39295 } }
I20260926 06:24:27.833109 26210 leader_election.cc:302] T 00000000000000000000000000000000 P d8cd7ba7344046118603015f10dc624f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8cd7ba7344046118603015f10dc624f; no voters:
I20260926 06:24:27.833276 26210 leader_election.cc:288] T 00000000000000000000000000000000 P d8cd7ba7344046118603015f10dc624f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:24:27.833313 26215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8cd7ba7344046118603015f10dc624f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:24:27.833505 26215 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8cd7ba7344046118603015f10dc624f [term 1 LEADER]: Becoming Leader. State: Replica: d8cd7ba7344046118603015f10dc624f, State: Running, Role: LEADER
I20260926 06:24:27.833541 26210 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8cd7ba7344046118603015f10dc624f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:24:27.833644 26215 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8cd7ba7344046118603015f10dc624f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8cd7ba7344046118603015f10dc624f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39295 } }
I20260926 06:24:27.833949 26215 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8cd7ba7344046118603015f10dc624f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8cd7ba7344046118603015f10dc624f. Latest consensus state: current_term: 1 leader_uuid: "d8cd7ba7344046118603015f10dc624f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8cd7ba7344046118603015f10dc624f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39295 } } }
I20260926 06:24:27.833961 26217 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8cd7ba7344046118603015f10dc624f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8cd7ba7344046118603015f10dc624f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8cd7ba7344046118603015f10dc624f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39295 } } }
I20260926 06:24:27.834079 26215 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8cd7ba7344046118603015f10dc624f [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:27.834169 26217 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8cd7ba7344046118603015f10dc624f [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:27.835237 26221 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:24:27.835789 26221 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:24:27.837412 26221 catalog_manager.cc:1347] Generated new cluster ID: 10257fee6eee4da9b9286264b9b0e52d
I20260926 06:24:27.837455 26221 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:24:27 dist-test-slave-pd0t krb5kdc[26129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 06:24:27.841723 26214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:27.830369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:36479 (local address 127.0.0.1:39295)
0926 06:24:27.830553 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:27.830560 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:27.830624 (+ 64us) server_negotiation.cc:407] Connection header received
0926 06:24:27.831732 (+ 1108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:27.831737 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:27.831783 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:27.831933 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:27.832725 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:27.834280 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:27.835094 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:27.835678 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:27.835782 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:27.838497 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:27.838501 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:27.838501 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:27.838515 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:27.840195 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:27.840592 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:27.840595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:27.840596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:27.840684 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:27.840927 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:27.840928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:27.840928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:27.841147 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:27.841300 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:27.841444 (+ 144us) server_negotiation.cc:299] Negotiation successful
0926 06:24:27.841545 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":104,"threads_started":1}
I20260926 06:24:27.842613 4906 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:24:27.843796 26221 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:24:27.844441 26221 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:24:27.850016 26214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:27.844090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:50205 (local address 127.0.0.1:39295)
0926 06:24:27.844123 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:27.844125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:27.844132 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:24:27.844151 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:27.844152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:27.844159 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:27.844202 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:27.844770 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:27.845227 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:27.845923 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:27.846278 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:27.846368 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:27.847504 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:27.847505 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:27.847506 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:27.847513 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:27.848870 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:27.849232 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:27.849233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:27.849233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:27.849283 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:27.849514 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:27.849514 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:27.849515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:27.849615 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:27.849715 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:27.849847 (+ 132us) server_negotiation.cc:299] Negotiation successful
0926 06:24:27.849913 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 06:24:27.850297 26221 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8cd7ba7344046118603015f10dc624f: Generated new TSK 0
I20260926 06:24:27.850492 26221 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:24:27.857240 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 26142
W20260926 06:24:27.860945 26127 connection.cc:535] client connection to 127.0.0.1:39295 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39295): Connection reset by peer (error 104)
2026-09-26T06:24:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1285 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:24:27 dist-test-slave-pd0t krb5kdc[26241](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:24:27 dist-test-slave-pd0t krb5kdc[26241](info): set up 2 sockets
Sep 26 06:24:27 dist-test-slave-pd0t krb5kdc[26241](info): commencing operation
krb5kdc: starting...
W20260926 06:24:28.882515 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:24:28 dist-test-slave-pd0t krb5kdc[26241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:24:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:24:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 06:24:28.993736 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37423
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37423
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:24:29.071882 26254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:29.072105 26254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:29.072141 26254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:29.073272 26254 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:24:29.073350 26254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:29.073376 26254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:29.073397 26254 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:24:29.073423 26254 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:24:29.074887 26254 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37423
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:29.075146 26254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:29.075363 26254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:29.080237 26260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:29.080256 26262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:29.080276 26259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:29.080518 26254 server_base.cc:1034] running on GCE node
I20260926 06:24:29.080814 26254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:29.081092 26254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:29.082258 26254 hybrid_clock.cc:648] HybridClock initialized: now 1790403869082233 us; error 36 us; skew 500 ppm
Sep 26 06:24:29 dist-test-slave-pd0t krb5kdc[26241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403869, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:29.084815 26254 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 06:24:29.085253 26254 webserver.cc:460] Webserver started at http://127.0.0.1:45605/ using document root <none> and password file <none>
I20260926 06:24:29.085454 26254 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:29.085494 26254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:29.085579 26254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:29.086549 26254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "0178b4191d12474380495e3f2ccbfba6"
format_stamp: "Formatted at 2026-09-26 06:24:29 on dist-test-slave-pd0t"
server_key: "b84887e9fe15b762cf71bbcbfc2ac16b"
server_key_iv: "252fbba6f8496339000bcdd2452d3ec3"
server_key_version: "encryptionkey@0"
I20260926 06:24:29.086916 26254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "0178b4191d12474380495e3f2ccbfba6"
format_stamp: "Formatted at 2026-09-26 06:24:29 on dist-test-slave-pd0t"
server_key: "b84887e9fe15b762cf71bbcbfc2ac16b"
server_key_iv: "252fbba6f8496339000bcdd2452d3ec3"
server_key_version: "encryptionkey@0"
I20260926 06:24:29.088217 26254 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:24:29.088912 26269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:29.089108 26254 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:24:29.089190 26254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "0178b4191d12474380495e3f2ccbfba6"
format_stamp: "Formatted at 2026-09-26 06:24:29 on dist-test-slave-pd0t"
server_key: "b84887e9fe15b762cf71bbcbfc2ac16b"
server_key_iv: "252fbba6f8496339000bcdd2452d3ec3"
server_key_version: "encryptionkey@0"
I20260926 06:24:29.089241 26254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:29.118964 26254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:29.121204 26254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:29.121328 26254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:29.125202 26254 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37423
I20260926 06:24:29.125262 26321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37423 every 8 connection(s)
I20260926 06:24:29.125593 26254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:24:29.126473 26322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:29.128039 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 26254
I20260926 06:24:29.128166 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:24:29.128374 4906 external_mini_cluster.cc:1428] Setting key 9262adc3d43f9d48e55b91e1d600eb41
I20260926 06:24:29.129133 26322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0178b4191d12474380495e3f2ccbfba6: Bootstrap starting.
I20260926 06:24:29.129719 26322 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0178b4191d12474380495e3f2ccbfba6: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:29.130069 26322 log.cc:824] T 00000000000000000000000000000000 P 0178b4191d12474380495e3f2ccbfba6: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:29.130808 26322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0178b4191d12474380495e3f2ccbfba6: No bootstrap required, opened a new log
I20260926 06:24:29.133136 26322 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0178b4191d12474380495e3f2ccbfba6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0178b4191d12474380495e3f2ccbfba6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37423 } }
I20260926 06:24:29.133327 26322 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0178b4191d12474380495e3f2ccbfba6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:29.133373 26322 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0178b4191d12474380495e3f2ccbfba6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0178b4191d12474380495e3f2ccbfba6, State: Initialized, Role: FOLLOWER
I20260926 06:24:29.133513 26322 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0178b4191d12474380495e3f2ccbfba6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0178b4191d12474380495e3f2ccbfba6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37423 } }
I20260926 06:24:29.133610 26322 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0178b4191d12474380495e3f2ccbfba6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:24:29.133654 26322 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0178b4191d12474380495e3f2ccbfba6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:24:29.133701 26322 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0178b4191d12474380495e3f2ccbfba6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:29.134073 26325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:29.128906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:49641 (local address 127.0.0.1:37423)
0926 06:24:29.129060 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:29.129073 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:29.129097 (+ 24us) server_negotiation.cc:407] Connection header received
0926 06:24:29.129567 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:29.129574 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:29.129622 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:29.129731 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:29.130008 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:29.131141 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:29.132242 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:29.132646 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:29.132734 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:29.133437 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:29.133441 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:29.133441 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:29.133643 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:29.133682 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:29.133697 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:29.133782 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:29.133870 (+ 88us) server_negotiation.cc:299] Negotiation successful
0926 06:24:29.133898 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":77,"threads_started":1}
I20260926 06:24:29.134397 26322 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0178b4191d12474380495e3f2ccbfba6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0178b4191d12474380495e3f2ccbfba6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37423 } }
I20260926 06:24:29.134516 26322 leader_election.cc:302] T 00000000000000000000000000000000 P 0178b4191d12474380495e3f2ccbfba6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0178b4191d12474380495e3f2ccbfba6; no voters:
I20260926 06:24:29.134691 26322 leader_election.cc:288] T 00000000000000000000000000000000 P 0178b4191d12474380495e3f2ccbfba6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:24:29.134740 26327 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0178b4191d12474380495e3f2ccbfba6 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:24:29.134896 26327 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0178b4191d12474380495e3f2ccbfba6 [term 1 LEADER]: Becoming Leader. State: Replica: 0178b4191d12474380495e3f2ccbfba6, State: Running, Role: LEADER
I20260926 06:24:29.135023 4906 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:24:29.134938 26322 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0178b4191d12474380495e3f2ccbfba6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:24:29.135053 26327 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0178b4191d12474380495e3f2ccbfba6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0178b4191d12474380495e3f2ccbfba6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37423 } }
I20260926 06:24:29.135563 26328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0178b4191d12474380495e3f2ccbfba6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0178b4191d12474380495e3f2ccbfba6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0178b4191d12474380495e3f2ccbfba6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37423 } } }
I20260926 06:24:29.135600 26329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0178b4191d12474380495e3f2ccbfba6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0178b4191d12474380495e3f2ccbfba6. Latest consensus state: current_term: 1 leader_uuid: "0178b4191d12474380495e3f2ccbfba6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0178b4191d12474380495e3f2ccbfba6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37423 } } }
I20260926 06:24:29.135643 26328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0178b4191d12474380495e3f2ccbfba6 [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:29.135673 26329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0178b4191d12474380495e3f2ccbfba6 [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:29.136134 26331 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:24:29.136735 26331 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:24:29.139432 26325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:29.136553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:38171 (local address 127.0.0.1:37423)
0926 06:24:29.136585 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:29.136587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:29.136596 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:24:29.136637 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:29.136638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:29.136648 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:29.136707 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:29.136903 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:29.137381 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:29.138120 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:29.138335 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:29.138422 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:29.139052 (+ 630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:29.139053 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:29.139053 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:29.139058 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:29.139071 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:29.139075 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:29.139132 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:29.139330 (+ 198us) server_negotiation.cc:299] Negotiation successful
0926 06:24:29.139339 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 06:24:29.139587 26331 catalog_manager.cc:1347] Generated new cluster ID: 3d51f497584a46f1ba3fd8b90b076afb
I20260926 06:24:29.139637 26331 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:24:29.154596 26331 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:24:29.155304 26331 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:24:29.161983 26331 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0178b4191d12474380495e3f2ccbfba6: Generated new TSK 0
I20260926 06:24:29.162140 26331 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:24:29.164283 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 26254
W20260926 06:24:29.168529 26239 connection.cc:535] client connection to 127.0.0.1:37423 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37423): Connection reset by peer (error 104)
2026-09-26T06:24:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1307 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:24:29 dist-test-slave-pd0t krb5kdc[26353](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:24:29 dist-test-slave-pd0t krb5kdc[26353](info): set up 2 sockets
Sep 26 06:24:29 dist-test-slave-pd0t krb5kdc[26353](info): commencing operation
krb5kdc: starting...
W20260926 06:24:30.203164 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:24:30 dist-test-slave-pd0t krb5kdc[26353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:24:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:24:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 06:24:30.323434 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40789
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40789
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:24:30.401105 26366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:30.401326 26366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:30.401360 26366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:30.402462 26366 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:24:30.402531 26366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:30.402558 26366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:30.402602 26366 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:24:30.402629 26366 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:24:30.404094 26366 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40789
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790403591640726-4906-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:30.404374 26366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:30.404585 26366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:30.409453 26374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:30.409447 26372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:30.409447 26371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:30.409618 26366 server_base.cc:1034] running on GCE node
I20260926 06:24:30.409921 26366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:30.410219 26366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:30.411379 26366 hybrid_clock.cc:648] HybridClock initialized: now 1790403870411358 us; error 41 us; skew 500 ppm
Sep 26 06:24:30 dist-test-slave-pd0t krb5kdc[26353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403870, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:30.413813 26366 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 06:24:30.414247 26366 webserver.cc:460] Webserver started at http://127.0.0.1:46061/ using document root <none> and password file <none>
I20260926 06:24:30.414477 26366 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:30.414546 26366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:30.414702 26366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:30.415676 26366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "305d290477194531a3315904ca9cc10c"
format_stamp: "Formatted at 2026-09-26 06:24:30 on dist-test-slave-pd0t"
server_key: "c918cfe451251819a13765eb72693219"
server_key_iv: "b3751f60c75c9521ae888817d5d90f88"
server_key_version: "encryptionkey@0"
I20260926 06:24:30.416033 26366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "305d290477194531a3315904ca9cc10c"
format_stamp: "Formatted at 2026-09-26 06:24:30 on dist-test-slave-pd0t"
server_key: "c918cfe451251819a13765eb72693219"
server_key_iv: "b3751f60c75c9521ae888817d5d90f88"
server_key_version: "encryptionkey@0"
I20260926 06:24:30.417303 26366 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 06:24:30.418068 26381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:30.418260 26366 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:24:30.418357 26366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "305d290477194531a3315904ca9cc10c"
format_stamp: "Formatted at 2026-09-26 06:24:30 on dist-test-slave-pd0t"
server_key: "c918cfe451251819a13765eb72693219"
server_key_iv: "b3751f60c75c9521ae888817d5d90f88"
server_key_version: "encryptionkey@0"
I20260926 06:24:30.418437 26366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:30.421656 26366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:30.423592 26366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:30.423745 26366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:30.427870 26366 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40789
I20260926 06:24:30.427917 26433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40789 every 8 connection(s)
I20260926 06:24:30.428228 26366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:24:30.429142 26434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:30.431344 26434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 305d290477194531a3315904ca9cc10c: Bootstrap starting.
I20260926 06:24:30.432057 26434 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 305d290477194531a3315904ca9cc10c: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:30.432381 26434 log.cc:824] T 00000000000000000000000000000000 P 305d290477194531a3315904ca9cc10c: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:30.433207 26434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 305d290477194531a3315904ca9cc10c: No bootstrap required, opened a new log
I20260926 06:24:30.435209 26434 raft_consensus.cc:348] T 00000000000000000000000000000000 P 305d290477194531a3315904ca9cc10c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "305d290477194531a3315904ca9cc10c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40789 } }
I20260926 06:24:30.435367 26434 raft_consensus.cc:374] T 00000000000000000000000000000000 P 305d290477194531a3315904ca9cc10c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:30.435391 26434 raft_consensus.cc:729] T 00000000000000000000000000000000 P 305d290477194531a3315904ca9cc10c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 305d290477194531a3315904ca9cc10c, State: Initialized, Role: FOLLOWER
I20260926 06:24:30.435484 26434 consensus_queue.cc:260] T 00000000000000000000000000000000 P 305d290477194531a3315904ca9cc10c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "305d290477194531a3315904ca9cc10c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40789 } }
I20260926 06:24:30.435540 26434 raft_consensus.cc:388] T 00000000000000000000000000000000 P 305d290477194531a3315904ca9cc10c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:24:30.435561 26434 raft_consensus.cc:482] T 00000000000000000000000000000000 P 305d290477194531a3315904ca9cc10c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:24:30.435591 26434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 305d290477194531a3315904ca9cc10c [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:30.436061 26434 raft_consensus.cc:504] T 00000000000000000000000000000000 P 305d290477194531a3315904ca9cc10c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "305d290477194531a3315904ca9cc10c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40789 } }
I20260926 06:24:30.436141 26434 leader_election.cc:302] T 00000000000000000000000000000000 P 305d290477194531a3315904ca9cc10c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 305d290477194531a3315904ca9cc10c; no voters:
I20260926 06:24:30.436264 26434 leader_election.cc:288] T 00000000000000000000000000000000 P 305d290477194531a3315904ca9cc10c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:24:30.436335 26437 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 305d290477194531a3315904ca9cc10c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:24:30.436492 26437 raft_consensus.cc:686] T 00000000000000000000000000000000 P 305d290477194531a3315904ca9cc10c [term 1 LEADER]: Becoming Leader. State: Replica: 305d290477194531a3315904ca9cc10c, State: Running, Role: LEADER
I20260926 06:24:30.436532 26434 sys_catalog.cc:564] T 00000000000000000000000000000000 P 305d290477194531a3315904ca9cc10c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:24:30.436609 26437 consensus_queue.cc:237] T 00000000000000000000000000000000 P 305d290477194531a3315904ca9cc10c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "305d290477194531a3315904ca9cc10c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40789 } }
I20260926 06:24:30.436988 26438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 305d290477194531a3315904ca9cc10c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "305d290477194531a3315904ca9cc10c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "305d290477194531a3315904ca9cc10c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40789 } } }
I20260926 06:24:30.436996 26439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 305d290477194531a3315904ca9cc10c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 305d290477194531a3315904ca9cc10c. Latest consensus state: current_term: 1 leader_uuid: "305d290477194531a3315904ca9cc10c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "305d290477194531a3315904ca9cc10c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40789 } } }
I20260926 06:24:30.437357 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 26366
I20260926 06:24:30.437495 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:24:30.437707 4906 external_mini_cluster.cc:1428] Setting key e332e5ce7b0f32338b1d4fc158431833
I20260926 06:24:30.437166 26438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 305d290477194531a3315904ca9cc10c [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:30.438167 26449 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:24:30.437207 26439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 305d290477194531a3315904ca9cc10c [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:30.438808 26449 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:24:30.440378 26449 catalog_manager.cc:1347] Generated new cluster ID: 48e702efa6494d39b64f8168c1a962c9
I20260926 06:24:30.440455 26449 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:24:30.440500 26455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:30.438166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:41491 (local address 127.0.0.1:40789)
0926 06:24:30.438324 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:30.438333 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:30.438612 (+ 279us) server_negotiation.cc:407] Connection header received
0926 06:24:30.438726 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:30.438731 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:30.438754 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:30.438906 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:30.440020 (+ 1114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:30.440024 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:30.440024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:30.440028 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:30.440068 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:30.440087 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:30.440160 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:30.440233 (+ 73us) server_negotiation.cc:299] Negotiation successful
0926 06:24:30.440278 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":63,"threads_started":1}
I20260926 06:24:30.441270 4906 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:24:30.447274 26455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:30.442534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:38009 (local address 127.0.0.1:40789)
0926 06:24:30.442568 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:30.442570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:30.442696 (+ 126us) server_negotiation.cc:407] Connection header received
0926 06:24:30.442764 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:30.442765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:30.442779 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:30.442825 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:30.447030 (+ 4205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:30.447031 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:30.447032 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:30.447036 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:30.447050 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:30.447054 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:30.447115 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:30.447168 (+ 53us) server_negotiation.cc:299] Negotiation successful
0926 06:24:30.447183 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 06:24:30.454680 26449 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:24:30.454835 26449 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:24:30.458782 26449 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 305d290477194531a3315904ca9cc10c: Generated new TSK 0
I20260926 06:24:30.458993 26449 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:24:30.463186 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 26366
2026-09-26T06:24:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1299 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:24:30 dist-test-slave-pd0t krb5kdc[26465](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:24:30 dist-test-slave-pd0t krb5kdc[26465](info): set up 2 sockets
Sep 26 06:24:30 dist-test-slave-pd0t krb5kdc[26465](info): commencing operation
krb5kdc: starting...
W20260926 06:24:31.478834 4906 mini_kdc.cc:121] Time spent starting KDC: real 0.998s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:24:31 dist-test-slave-pd0t krb5kdc[26465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:24:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:24:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.202.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
I20260926 06:24:31.590857 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:36983
--webserver_interface=127.4.202.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:41521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:36983
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:24:31.668051 26478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:31.668273 26478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:31.668313 26478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:31.669379 26478 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:24:31.669447 26478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:31.669469 26478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:31.669495 26478 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:24:31.669521 26478 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:24:31.670878 26478 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:41521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:36983
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:36983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:31.671139 26478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:31.671383 26478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:31.676149 26484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:31.676159 26483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:31.676172 26486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:31.676332 26478 server_base.cc:1034] running on GCE node
I20260926 06:24:31.676644 26478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:31.676916 26478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:31.678110 26478 hybrid_clock.cc:648] HybridClock initialized: now 1790403871678105 us; error 37 us; skew 500 ppm
Sep 26 06:24:31 dist-test-slave-pd0t krb5kdc[26465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403871, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:31.680639 26478 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:24:31.681115 26478 webserver.cc:460] Webserver started at http://127.4.202.190:36923/ using document root <none> and password file <none>
I20260926 06:24:31.681392 26478 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:31.681468 26478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:31.681602 26478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:31.682690 26478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "de7afe92437a4d938f2049b3512b2d71"
format_stamp: "Formatted at 2026-09-26 06:24:31 on dist-test-slave-pd0t"
server_key: "9ed089617cad54ff601e0d573bc8313b"
server_key_iv: "9ca4392c6136496d0f345f9fa57776c2"
server_key_version: "encryptionkey@0"
I20260926 06:24:31.683064 26478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "de7afe92437a4d938f2049b3512b2d71"
format_stamp: "Formatted at 2026-09-26 06:24:31 on dist-test-slave-pd0t"
server_key: "9ed089617cad54ff601e0d573bc8313b"
server_key_iv: "9ca4392c6136496d0f345f9fa57776c2"
server_key_version: "encryptionkey@0"
I20260926 06:24:31.684391 26478 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:24:31.685107 26493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:31.685300 26478 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:24:31.685406 26478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "de7afe92437a4d938f2049b3512b2d71"
format_stamp: "Formatted at 2026-09-26 06:24:31 on dist-test-slave-pd0t"
server_key: "9ed089617cad54ff601e0d573bc8313b"
server_key_iv: "9ca4392c6136496d0f345f9fa57776c2"
server_key_version: "encryptionkey@0"
I20260926 06:24:31.685492 26478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:31.698778 26478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:31.701280 26478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:31.701442 26478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:31.706908 26478 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:36983
I20260926 06:24:31.706964 26545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:36983 every 8 connection(s)
I20260926 06:24:31.707397 26478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:24:31.708230 26546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:31.710896 26546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de7afe92437a4d938f2049b3512b2d71: Bootstrap starting.
I20260926 06:24:31.711464 26546 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de7afe92437a4d938f2049b3512b2d71: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:31.711829 26546 log.cc:824] T 00000000000000000000000000000000 P de7afe92437a4d938f2049b3512b2d71: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:31.712525 26546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de7afe92437a4d938f2049b3512b2d71: No bootstrap required, opened a new log
I20260926 06:24:31.714057 26546 raft_consensus.cc:348] T 00000000000000000000000000000000 P de7afe92437a4d938f2049b3512b2d71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de7afe92437a4d938f2049b3512b2d71" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 36983 } }
I20260926 06:24:31.714217 26546 raft_consensus.cc:374] T 00000000000000000000000000000000 P de7afe92437a4d938f2049b3512b2d71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:31.714265 26546 raft_consensus.cc:729] T 00000000000000000000000000000000 P de7afe92437a4d938f2049b3512b2d71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de7afe92437a4d938f2049b3512b2d71, State: Initialized, Role: FOLLOWER
I20260926 06:24:31.714425 26546 consensus_queue.cc:260] T 00000000000000000000000000000000 P de7afe92437a4d938f2049b3512b2d71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de7afe92437a4d938f2049b3512b2d71" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 36983 } }
I20260926 06:24:31.714525 26546 raft_consensus.cc:388] T 00000000000000000000000000000000 P de7afe92437a4d938f2049b3512b2d71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:24:31.714568 26546 raft_consensus.cc:482] T 00000000000000000000000000000000 P de7afe92437a4d938f2049b3512b2d71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:24:31.714716 26546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de7afe92437a4d938f2049b3512b2d71 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:31.714977 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 26478
I20260926 06:24:31.715087 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:24:31.715272 4906 external_mini_cluster.cc:1428] Setting key b4faa34b56877ed54a34277d11e21b11
I20260926 06:24:31.715874 26546 raft_consensus.cc:504] T 00000000000000000000000000000000 P de7afe92437a4d938f2049b3512b2d71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de7afe92437a4d938f2049b3512b2d71" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 36983 } }
I20260926 06:24:31.715979 26546 leader_election.cc:302] T 00000000000000000000000000000000 P de7afe92437a4d938f2049b3512b2d71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de7afe92437a4d938f2049b3512b2d71; no voters:
I20260926 06:24:31.716130 26546 leader_election.cc:288] T 00000000000000000000000000000000 P de7afe92437a4d938f2049b3512b2d71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:24:31.716184 26551 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de7afe92437a4d938f2049b3512b2d71 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:24:31.716361 26551 raft_consensus.cc:686] T 00000000000000000000000000000000 P de7afe92437a4d938f2049b3512b2d71 [term 1 LEADER]: Becoming Leader. State: Replica: de7afe92437a4d938f2049b3512b2d71, State: Running, Role: LEADER
I20260926 06:24:31.716406 26546 sys_catalog.cc:564] T 00000000000000000000000000000000 P de7afe92437a4d938f2049b3512b2d71 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:24:31.716449 26551 consensus_queue.cc:237] T 00000000000000000000000000000000 P de7afe92437a4d938f2049b3512b2d71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de7afe92437a4d938f2049b3512b2d71" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 36983 } }
I20260926 06:24:31.716764 26551 sys_catalog.cc:455] T 00000000000000000000000000000000 P de7afe92437a4d938f2049b3512b2d71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de7afe92437a4d938f2049b3512b2d71. Latest consensus state: current_term: 1 leader_uuid: "de7afe92437a4d938f2049b3512b2d71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de7afe92437a4d938f2049b3512b2d71" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 36983 } } }
I20260926 06:24:31.716909 26551 sys_catalog.cc:458] T 00000000000000000000000000000000 P de7afe92437a4d938f2049b3512b2d71 [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:31.717166 26559 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:24:31.717440 26552 sys_catalog.cc:455] T 00000000000000000000000000000000 P de7afe92437a4d938f2049b3512b2d71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de7afe92437a4d938f2049b3512b2d71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de7afe92437a4d938f2049b3512b2d71" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 36983 } } }
I20260926 06:24:31.717569 26552 sys_catalog.cc:458] T 00000000000000000000000000000000 P de7afe92437a4d938f2049b3512b2d71 [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:31.717792 26559 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:24:31.719769 26559 catalog_manager.cc:1347] Generated new cluster ID: e25436b083844335b691ced22b71359c
I20260926 06:24:31.719848 26559 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 06:24:31 dist-test-slave-pd0t krb5kdc[26465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:24:31.727039 26550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:31.715768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49291 (local address 127.4.202.190:36983)
0926 06:24:31.715942 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:31.715952 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:31.715994 (+ 42us) server_negotiation.cc:407] Connection header received
0926 06:24:31.716100 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:31.716105 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:31.716146 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:31.716291 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:31.717241 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:31.718792 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:31.719742 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:31.720356 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:31.720465 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:31.723080 (+ 2615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:31.723084 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:31.723084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:31.723096 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:31.725192 (+ 2096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:31.725619 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:31.725622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:31.725622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:31.725743 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:31.725980 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:31.725981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:31.725982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:31.726282 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:31.726471 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:31.726552 (+ 81us) server_negotiation.cc:299] Negotiation successful
0926 06:24:31.726817 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
I20260926 06:24:31.728008 4906 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:24:31.736398 26550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:31.729413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38115 (local address 127.4.202.190:36983)
0926 06:24:31.729451 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:31.729453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:31.729561 (+ 108us) server_negotiation.cc:407] Connection header received
0926 06:24:31.729585 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:31.729586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:31.729596 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:31.729665 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:31.730212 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:31.730961 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:31.731898 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:31.732238 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:31.732309 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:31.733592 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:31.733594 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:31.733594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:31.733600 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:31.735184 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:31.735629 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:31.735630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:31.735631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:31.735687 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:31.735911 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:31.735912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:31.735912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:31.736032 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:31.736142 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:31.736247 (+ 105us) server_negotiation.cc:299] Negotiation successful
0926 06:24:31.736310 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 06:24:31.749430 26559 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:24:31.749902 26559 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:24:31.757130 26559 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de7afe92437a4d938f2049b3512b2d71: Generated new TSK 0
I20260926 06:24:31.757318 26559 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:24:31.763317 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 26478
W20260926 06:24:31.767715 26463 connection.cc:535] client connection to 127.4.202.190:36983 recv error: Network error: failed to read from TLS socket (remote: 127.4.202.190:36983): Connection reset by peer (error 104)
2026-09-26T06:24:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1300 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:24:31 dist-test-slave-pd0t krb5kdc[26577](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:24:31 dist-test-slave-pd0t krb5kdc[26577](info): set up 2 sockets
Sep 26 06:24:31 dist-test-slave-pd0t krb5kdc[26577](info): commencing operation
krb5kdc: starting...
W20260926 06:24:32.786698 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:24:32 dist-test-slave-pd0t krb5kdc[26577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:24:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:24:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.202.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
I20260926 06:24:32.897756 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:44851
--webserver_interface=127.4.202.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:39701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:44851
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:24:32.975076 26590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:32.975315 26590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:32.975353 26590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:32.976584 26590 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:24:32.976647 26590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:32.976675 26590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:32.976701 26590 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:24:32.976723 26590 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:24:32.978147 26590 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:39701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:44851
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:44851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:32.978410 26590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:32.978688 26590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:32.983407 26595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:32.983388 26596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:32.983389 26598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:32.983695 26590 server_base.cc:1034] running on GCE node
I20260926 06:24:32.983914 26590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:32.984180 26590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:32.985340 26590 hybrid_clock.cc:648] HybridClock initialized: now 1790403872985329 us; error 31 us; skew 500 ppm
Sep 26 06:24:32 dist-test-slave-pd0t krb5kdc[26577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403872, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:32.991101 26590 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:24:32.991603 26590 webserver.cc:460] Webserver started at http://127.4.202.190:43927/ using document root <none> and password file <none>
I20260926 06:24:32.991833 26590 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:32.991900 26590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:32.992036 26590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:32.993021 26590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "4b17476c9d464d24a533a06b2d1db0a1"
format_stamp: "Formatted at 2026-09-26 06:24:32 on dist-test-slave-pd0t"
server_key: "45ccaf578c4325ac6495415ad23cef19"
server_key_iv: "b80e14d5fa73c30162af2e1c7bc31d42"
server_key_version: "encryptionkey@0"
I20260926 06:24:32.993376 26590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "4b17476c9d464d24a533a06b2d1db0a1"
format_stamp: "Formatted at 2026-09-26 06:24:32 on dist-test-slave-pd0t"
server_key: "45ccaf578c4325ac6495415ad23cef19"
server_key_iv: "b80e14d5fa73c30162af2e1c7bc31d42"
server_key_version: "encryptionkey@0"
I20260926 06:24:32.994729 26590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:24:32.995486 26605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:32.995666 26590 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:24:32.995760 26590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "4b17476c9d464d24a533a06b2d1db0a1"
format_stamp: "Formatted at 2026-09-26 06:24:32 on dist-test-slave-pd0t"
server_key: "45ccaf578c4325ac6495415ad23cef19"
server_key_iv: "b80e14d5fa73c30162af2e1c7bc31d42"
server_key_version: "encryptionkey@0"
I20260926 06:24:32.995846 26590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:33.008947 26590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:33.010679 26590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:33.010818 26590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:33.014864 26590 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:44851
I20260926 06:24:33.014897 26657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:44851 every 8 connection(s)
I20260926 06:24:33.015209 26590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:24:33.016052 26658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:33.018090 26658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b17476c9d464d24a533a06b2d1db0a1: Bootstrap starting.
I20260926 06:24:33.018757 26658 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b17476c9d464d24a533a06b2d1db0a1: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:33.019125 26658 log.cc:824] T 00000000000000000000000000000000 P 4b17476c9d464d24a533a06b2d1db0a1: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:33.019922 26658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b17476c9d464d24a533a06b2d1db0a1: No bootstrap required, opened a new log
I20260926 06:24:33.021430 26658 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b17476c9d464d24a533a06b2d1db0a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b17476c9d464d24a533a06b2d1db0a1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 44851 } }
I20260926 06:24:33.021555 26658 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b17476c9d464d24a533a06b2d1db0a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:33.021576 26658 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b17476c9d464d24a533a06b2d1db0a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b17476c9d464d24a533a06b2d1db0a1, State: Initialized, Role: FOLLOWER
I20260926 06:24:33.021668 26658 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b17476c9d464d24a533a06b2d1db0a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b17476c9d464d24a533a06b2d1db0a1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 44851 } }
I20260926 06:24:33.021716 26658 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b17476c9d464d24a533a06b2d1db0a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:24:33.021736 26658 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b17476c9d464d24a533a06b2d1db0a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:24:33.021806 26658 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b17476c9d464d24a533a06b2d1db0a1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:33.021947 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 26590
I20260926 06:24:33.022066 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:24:33.022233 4906 external_mini_cluster.cc:1428] Setting key 6fe6857da6690f864ebf6b70f816c533
I20260926 06:24:33.022950 26658 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b17476c9d464d24a533a06b2d1db0a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b17476c9d464d24a533a06b2d1db0a1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 44851 } }
I20260926 06:24:33.023048 26658 leader_election.cc:302] T 00000000000000000000000000000000 P 4b17476c9d464d24a533a06b2d1db0a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b17476c9d464d24a533a06b2d1db0a1; no voters:
I20260926 06:24:33.023223 26658 leader_election.cc:288] T 00000000000000000000000000000000 P 4b17476c9d464d24a533a06b2d1db0a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:24:33.023259 26663 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b17476c9d464d24a533a06b2d1db0a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:24:33.023475 26663 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b17476c9d464d24a533a06b2d1db0a1 [term 1 LEADER]: Becoming Leader. State: Replica: 4b17476c9d464d24a533a06b2d1db0a1, State: Running, Role: LEADER
I20260926 06:24:33.023547 26658 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b17476c9d464d24a533a06b2d1db0a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:24:33.023593 26663 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b17476c9d464d24a533a06b2d1db0a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b17476c9d464d24a533a06b2d1db0a1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 44851 } }
I20260926 06:24:33.023892 26663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b17476c9d464d24a533a06b2d1db0a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b17476c9d464d24a533a06b2d1db0a1. Latest consensus state: current_term: 1 leader_uuid: "4b17476c9d464d24a533a06b2d1db0a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b17476c9d464d24a533a06b2d1db0a1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 44851 } } }
I20260926 06:24:33.023983 26663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b17476c9d464d24a533a06b2d1db0a1 [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:33.023900 26664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b17476c9d464d24a533a06b2d1db0a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b17476c9d464d24a533a06b2d1db0a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b17476c9d464d24a533a06b2d1db0a1" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 44851 } } }
I20260926 06:24:33.024114 26664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b17476c9d464d24a533a06b2d1db0a1 [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:33.024809 26670 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:24:33.025418 26670 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:24:33.026933 26670 catalog_manager.cc:1347] Generated new cluster ID: 9c65b5b40cc7437498dec1e9a1e07aef
I20260926 06:24:33.026990 26670 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:24:33.027988 26662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:33.022710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36201 (local address 127.4.202.190:44851)
0926 06:24:33.022925 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:33.022933 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:33.022968 (+ 35us) server_negotiation.cc:407] Connection header received
0926 06:24:33.023099 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:33.023105 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:33.023156 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:33.023296 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:33.023557 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:33.025053 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:33.025895 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:33.026497 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:33.026616 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:33.027392 (+ 776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:33.027395 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:33.027396 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:33.027560 (+ 164us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:33.027588 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:33.027598 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:33.027690 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:33.027765 (+ 75us) server_negotiation.cc:299] Negotiation successful
0926 06:24:33.027791 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":139,"threads_started":1}
I20260926 06:24:33.028760 4906 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:24:33.032884 26662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:33.029965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56111 (local address 127.4.202.190:44851)
0926 06:24:33.030016 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:33.030021 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:33.030138 (+ 117us) server_negotiation.cc:407] Connection header received
0926 06:24:33.030225 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:33.030225 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:33.030235 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:33.030286 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:33.030428 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:33.030992 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:33.031702 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:33.031944 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:33.032018 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:33.032652 (+ 634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:33.032653 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:33.032654 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:33.032658 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:33.032669 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:33.032673 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:33.032725 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:33.032784 (+ 59us) server_negotiation.cc:299] Negotiation successful
0926 06:24:33.032796 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 06:24:33.046554 26670 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:24:33.047232 26670 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:24:33.050664 26670 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b17476c9d464d24a533a06b2d1db0a1: Generated new TSK 0
I20260926 06:24:33.050824 26670 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:24:33.054703 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 26590
W20260926 06:24:33.058781 26575 connection.cc:535] client connection to 127.4.202.190:44851 recv error: Network error: failed to read from TLS socket (remote: 127.4.202.190:44851): Connection reset by peer (error 104)
2026-09-26T06:24:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1291 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:24:33 dist-test-slave-pd0t krb5kdc[26689](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:24:33 dist-test-slave-pd0t krb5kdc[26689](info): set up 2 sockets
Sep 26 06:24:33 dist-test-slave-pd0t krb5kdc[26689](info): commencing operation
krb5kdc: starting...
W20260926 06:24:34.090507 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:24:34 dist-test-slave-pd0t krb5kdc[26689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:24:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:24:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.202.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
I20260926 06:24:34.205570 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:34773
--webserver_interface=127.4.202.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:40277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:34773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:24:34.283973 26702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:34.284158 26702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:34.284181 26702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:34.285353 26702 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:24:34.285398 26702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:34.285413 26702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:34.285424 26702 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:24:34.285436 26702 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:24:34.286919 26702 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:40277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:34773
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:34773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:34.287158 26702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:34.287389 26702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:34.292295 26707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:34.292295 26710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:34.292357 26708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:34.292357 26702 server_base.cc:1034] running on GCE node
I20260926 06:24:34.292732 26702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:34.293023 26702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:34.294193 26702 hybrid_clock.cc:648] HybridClock initialized: now 1790403874294179 us; error 44 us; skew 500 ppm
Sep 26 06:24:34 dist-test-slave-pd0t krb5kdc[26689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403874, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:34.296736 26702 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:24:34.297166 26702 webserver.cc:460] Webserver started at http://127.4.202.190:40403/ using document root <none> and password file <none>
I20260926 06:24:34.297402 26702 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:34.297472 26702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:34.297616 26702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:34.298702 26702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "7b033524a32346fdba1060d19c8b70db"
format_stamp: "Formatted at 2026-09-26 06:24:34 on dist-test-slave-pd0t"
server_key: "41949301d9d0d369d0291ee58f1c8884"
server_key_iv: "760933f09bc0dcc92a2bf976ba22e8a9"
server_key_version: "encryptionkey@0"
I20260926 06:24:34.299054 26702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "7b033524a32346fdba1060d19c8b70db"
format_stamp: "Formatted at 2026-09-26 06:24:34 on dist-test-slave-pd0t"
server_key: "41949301d9d0d369d0291ee58f1c8884"
server_key_iv: "760933f09bc0dcc92a2bf976ba22e8a9"
server_key_version: "encryptionkey@0"
I20260926 06:24:34.300379 26702 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:24:34.301059 26717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:34.301241 26702 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:24:34.301338 26702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "7b033524a32346fdba1060d19c8b70db"
format_stamp: "Formatted at 2026-09-26 06:24:34 on dist-test-slave-pd0t"
server_key: "41949301d9d0d369d0291ee58f1c8884"
server_key_iv: "760933f09bc0dcc92a2bf976ba22e8a9"
server_key_version: "encryptionkey@0"
I20260926 06:24:34.301417 26702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:34.304538 26702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:34.306157 26702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:34.306322 26702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:34.310161 26702 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:34773
I20260926 06:24:34.310194 26769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:34773 every 8 connection(s)
I20260926 06:24:34.310490 26702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:24:34.311458 26770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:34.314069 26770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b033524a32346fdba1060d19c8b70db: Bootstrap starting.
I20260926 06:24:34.314702 26770 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b033524a32346fdba1060d19c8b70db: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:34.314996 26770 log.cc:824] T 00000000000000000000000000000000 P 7b033524a32346fdba1060d19c8b70db: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:34.315650 26770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b033524a32346fdba1060d19c8b70db: No bootstrap required, opened a new log
I20260926 06:24:34.317201 26770 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b033524a32346fdba1060d19c8b70db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b033524a32346fdba1060d19c8b70db" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34773 } }
I20260926 06:24:34.317330 26770 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b033524a32346fdba1060d19c8b70db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:34.317351 26770 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b033524a32346fdba1060d19c8b70db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b033524a32346fdba1060d19c8b70db, State: Initialized, Role: FOLLOWER
I20260926 06:24:34.317543 26770 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b033524a32346fdba1060d19c8b70db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b033524a32346fdba1060d19c8b70db" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34773 } }
I20260926 06:24:34.317605 26770 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b033524a32346fdba1060d19c8b70db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:24:34.317626 26770 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b033524a32346fdba1060d19c8b70db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:24:34.317698 26770 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b033524a32346fdba1060d19c8b70db [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:34.318507 26770 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b033524a32346fdba1060d19c8b70db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b033524a32346fdba1060d19c8b70db" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34773 } }
I20260926 06:24:34.318650 26770 leader_election.cc:302] T 00000000000000000000000000000000 P 7b033524a32346fdba1060d19c8b70db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b033524a32346fdba1060d19c8b70db; no voters:
I20260926 06:24:34.318818 26770 leader_election.cc:288] T 00000000000000000000000000000000 P 7b033524a32346fdba1060d19c8b70db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:24:34.318897 26773 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b033524a32346fdba1060d19c8b70db [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:24:34.319058 26773 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b033524a32346fdba1060d19c8b70db [term 1 LEADER]: Becoming Leader. State: Replica: 7b033524a32346fdba1060d19c8b70db, State: Running, Role: LEADER
I20260926 06:24:34.319098 26770 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b033524a32346fdba1060d19c8b70db [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:24:34.319165 26773 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b033524a32346fdba1060d19c8b70db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b033524a32346fdba1060d19c8b70db" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34773 } }
I20260926 06:24:34.319556 26774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b033524a32346fdba1060d19c8b70db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b033524a32346fdba1060d19c8b70db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b033524a32346fdba1060d19c8b70db" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34773 } } }
I20260926 06:24:34.319679 26774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b033524a32346fdba1060d19c8b70db [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:34.319705 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 26702
I20260926 06:24:34.319833 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:24:34.320024 4906 external_mini_cluster.cc:1428] Setting key 6bbeb92bf3faf943fa0334cfa536a2ae
I20260926 06:24:34.320786 26775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b033524a32346fdba1060d19c8b70db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b033524a32346fdba1060d19c8b70db. Latest consensus state: current_term: 1 leader_uuid: "7b033524a32346fdba1060d19c8b70db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b033524a32346fdba1060d19c8b70db" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 34773 } } }
I20260926 06:24:34.320875 26775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b033524a32346fdba1060d19c8b70db [sys.catalog]: This master's current role is: LEADER
W20260926 06:24:34.320848 26789 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7b033524a32346fdba1060d19c8b70db: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:24:34.320920 26789 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:24:34.321182 26782 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:24:34.321363 26782 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:24:34.322997 26782 catalog_manager.cc:1347] Generated new cluster ID: 2fdab87f387b45ef97636534d6531cd9
I20260926 06:24:34.323047 26782 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:24:34.323757 26791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:34.320429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46665 (local address 127.4.202.190:34773)
0926 06:24:34.322241 (+ 1812us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:34.322249 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:34.322282 (+ 33us) server_negotiation.cc:407] Connection header received
0926 06:24:34.322411 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:34.322417 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:34.322446 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:34.322607 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:34.323341 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:34.323345 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:34.323345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:34.323349 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:34.323403 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:34.323414 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:34.323491 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:34.323566 (+ 75us) server_negotiation.cc:299] Negotiation successful
0926 06:24:34.323607 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1765,"thread_start_us":63,"threads_started":1}
I20260926 06:24:34.324576 4906 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:24:34.327200 26791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:34.326042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59745 (local address 127.4.202.190:34773)
0926 06:24:34.326093 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:34.326095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:34.326247 (+ 152us) server_negotiation.cc:407] Connection header received
0926 06:24:34.326329 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:34.326330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:34.326335 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:34.326404 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:34.326991 (+ 587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:34.326992 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:34.326993 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:34.326997 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:34.327009 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:34.327013 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:34.327070 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:34.327126 (+ 56us) server_negotiation.cc:299] Negotiation successful
0926 06:24:34.327140 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 06:24:34.343189 26782 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:24:34.343366 26782 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:24:34.350911 26782 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b033524a32346fdba1060d19c8b70db: Generated new TSK 0
I20260926 06:24:34.351091 26782 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:24:34.354022 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 26702
2026-09-26T06:24:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1299 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:24:34 dist-test-slave-pd0t krb5kdc[26801](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:24:34 dist-test-slave-pd0t krb5kdc[26801](info): set up 2 sockets
Sep 26 06:24:34 dist-test-slave-pd0t krb5kdc[26801](info): commencing operation
krb5kdc: starting...
W20260926 06:24:35.386705 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:24:35 dist-test-slave-pd0t krb5kdc[26801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403875, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:24:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:24:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.202.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
I20260926 06:24:35.503834 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:40275
--webserver_interface=127.4.202.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:35155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:40275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:24:35.583292 26814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:35.583474 26814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:35.583496 26814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:35.584571 26814 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:24:35.584604 26814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:35.584617 26814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:35.584630 26814 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:24:35.584641 26814 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:24:35.586774 26814 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:35155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:40275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:40275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:35.586952 26814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:35.587107 26814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:35.591914 26820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:35.591980 26814 server_base.cc:1034] running on GCE node
W20260926 06:24:35.591914 26822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:35.591892 26819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:35.592329 26814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:35.592603 26814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:35.593787 26814 hybrid_clock.cc:648] HybridClock initialized: now 1790403875593767 us; error 37 us; skew 500 ppm
Sep 26 06:24:35 dist-test-slave-pd0t krb5kdc[26801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403875, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:35.596314 26814 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:24:35.596681 26814 webserver.cc:460] Webserver started at http://127.4.202.190:36871/ using document root <none> and password file <none>
I20260926 06:24:35.596868 26814 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:35.596907 26814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:35.596992 26814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:35.597911 26814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "a649338b5d554bf9ae233b158bdba2a2"
format_stamp: "Formatted at 2026-09-26 06:24:35 on dist-test-slave-pd0t"
server_key: "876696da054d0ece938dd51ce16e84d5"
server_key_iv: "981184069f0d35be95117bcafcaba7e9"
server_key_version: "encryptionkey@0"
I20260926 06:24:35.598188 26814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "a649338b5d554bf9ae233b158bdba2a2"
format_stamp: "Formatted at 2026-09-26 06:24:35 on dist-test-slave-pd0t"
server_key: "876696da054d0ece938dd51ce16e84d5"
server_key_iv: "981184069f0d35be95117bcafcaba7e9"
server_key_version: "encryptionkey@0"
I20260926 06:24:35.599509 26814 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:24:35.600167 26829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:35.600348 26814 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:24:35.600448 26814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "a649338b5d554bf9ae233b158bdba2a2"
format_stamp: "Formatted at 2026-09-26 06:24:35 on dist-test-slave-pd0t"
server_key: "876696da054d0ece938dd51ce16e84d5"
server_key_iv: "981184069f0d35be95117bcafcaba7e9"
server_key_version: "encryptionkey@0"
I20260926 06:24:35.600529 26814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:35.630450 26814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:35.633092 26814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:35.633270 26814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:35.637331 26814 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:40275
I20260926 06:24:35.637382 26881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:40275 every 8 connection(s)
I20260926 06:24:35.637734 26814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:24:35.638217 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 26814
I20260926 06:24:35.638335 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:24:35.638516 4906 external_mini_cluster.cc:1428] Setting key ad4cbcf02f6724e4b9a7ff36cb44aeff
I20260926 06:24:35.638756 26882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:35.642084 26882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a649338b5d554bf9ae233b158bdba2a2: Bootstrap starting.
I20260926 06:24:35.642743 26882 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a649338b5d554bf9ae233b158bdba2a2: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:35.643075 26882 log.cc:824] T 00000000000000000000000000000000 P a649338b5d554bf9ae233b158bdba2a2: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:35.643869 26882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a649338b5d554bf9ae233b158bdba2a2: No bootstrap required, opened a new log
Sep 26 06:24:35 dist-test-slave-pd0t krb5kdc[26801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403875, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:24:35.646080 26882 raft_consensus.cc:348] T 00000000000000000000000000000000 P a649338b5d554bf9ae233b158bdba2a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a649338b5d554bf9ae233b158bdba2a2" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 40275 } }
I20260926 06:24:35.646252 26882 raft_consensus.cc:374] T 00000000000000000000000000000000 P a649338b5d554bf9ae233b158bdba2a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:35.646281 26882 raft_consensus.cc:729] T 00000000000000000000000000000000 P a649338b5d554bf9ae233b158bdba2a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a649338b5d554bf9ae233b158bdba2a2, State: Initialized, Role: FOLLOWER
I20260926 06:24:35.646440 26882 consensus_queue.cc:260] T 00000000000000000000000000000000 P a649338b5d554bf9ae233b158bdba2a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a649338b5d554bf9ae233b158bdba2a2" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 40275 } }
I20260926 06:24:35.646553 26882 raft_consensus.cc:388] T 00000000000000000000000000000000 P a649338b5d554bf9ae233b158bdba2a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:24:35.646613 26882 raft_consensus.cc:482] T 00000000000000000000000000000000 P a649338b5d554bf9ae233b158bdba2a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:24:35.646715 26882 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a649338b5d554bf9ae233b158bdba2a2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:35.647387 26882 raft_consensus.cc:504] T 00000000000000000000000000000000 P a649338b5d554bf9ae233b158bdba2a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a649338b5d554bf9ae233b158bdba2a2" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 40275 } }
I20260926 06:24:35.647516 26882 leader_election.cc:302] T 00000000000000000000000000000000 P a649338b5d554bf9ae233b158bdba2a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a649338b5d554bf9ae233b158bdba2a2; no voters:
I20260926 06:24:35.647675 26882 leader_election.cc:288] T 00000000000000000000000000000000 P a649338b5d554bf9ae233b158bdba2a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:24:35.647707 26887 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a649338b5d554bf9ae233b158bdba2a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:24:35.647876 26887 raft_consensus.cc:686] T 00000000000000000000000000000000 P a649338b5d554bf9ae233b158bdba2a2 [term 1 LEADER]: Becoming Leader. State: Replica: a649338b5d554bf9ae233b158bdba2a2, State: Running, Role: LEADER
I20260926 06:24:35.647995 26882 sys_catalog.cc:564] T 00000000000000000000000000000000 P a649338b5d554bf9ae233b158bdba2a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:24:35.648005 26887 consensus_queue.cc:237] T 00000000000000000000000000000000 P a649338b5d554bf9ae233b158bdba2a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a649338b5d554bf9ae233b158bdba2a2" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 40275 } }
I20260926 06:24:35.648728 26887 sys_catalog.cc:455] T 00000000000000000000000000000000 P a649338b5d554bf9ae233b158bdba2a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a649338b5d554bf9ae233b158bdba2a2. Latest consensus state: current_term: 1 leader_uuid: "a649338b5d554bf9ae233b158bdba2a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a649338b5d554bf9ae233b158bdba2a2" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 40275 } } }
I20260926 06:24:35.648881 26887 sys_catalog.cc:458] T 00000000000000000000000000000000 P a649338b5d554bf9ae233b158bdba2a2 [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:35.649004 26888 sys_catalog.cc:455] T 00000000000000000000000000000000 P a649338b5d554bf9ae233b158bdba2a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a649338b5d554bf9ae233b158bdba2a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a649338b5d554bf9ae233b158bdba2a2" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 40275 } } }
I20260926 06:24:35.649080 26888 sys_catalog.cc:458] T 00000000000000000000000000000000 P a649338b5d554bf9ae233b158bdba2a2 [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:35.649181 26896 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:24:35.649869 26896 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:24:35.650130 26885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:35.639097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36295 (local address 127.4.202.190:40275)
0926 06:24:35.639328 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:35.639337 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:35.639368 (+ 31us) server_negotiation.cc:407] Connection header received
0926 06:24:35.639953 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:35.639961 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:35.640024 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:35.640177 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:35.640752 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:35.642075 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:35.642926 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:35.643469 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:35.643648 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:35.646185 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:35.646189 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:35.646189 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:35.646198 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:35.648193 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:35.648617 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:35.648620 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:35.648620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:35.648719 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:35.648951 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:35.648951 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:35.648952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:35.649255 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:35.649418 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:35.649785 (+ 367us) server_negotiation.cc:299] Negotiation successful
0926 06:24:35.649934 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":104,"threads_started":1}
I20260926 06:24:35.651183 4906 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:24:35.651467 26896 catalog_manager.cc:1347] Generated new cluster ID: 0fc68e13fc724a64bbfa142b40d27499
I20260926 06:24:35.651521 26896 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:24:35.664983 26885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:35.658944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50307 (local address 127.4.202.190:40275)
0926 06:24:35.658982 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:35.658984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:35.659118 (+ 134us) server_negotiation.cc:407] Connection header received
0926 06:24:35.659164 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:35.659165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:35.659175 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:35.659251 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:35.659782 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:35.660249 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:35.660942 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:35.661160 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:35.661248 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:35.662412 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:35.662413 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:35.662414 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:35.662421 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:35.663835 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:35.664208 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:35.664209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:35.664209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:35.664265 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:35.664479 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:35.664480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:35.664480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:35.664589 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:35.664691 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:35.664799 (+ 108us) server_negotiation.cc:299] Negotiation successful
0926 06:24:35.664871 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 06:24:35.675539 26896 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:24:35.676014 26896 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:24:35.686606 26896 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a649338b5d554bf9ae233b158bdba2a2: Generated new TSK 0
I20260926 06:24:35.686796 26896 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:24:35.690279 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 26814
2026-09-26T06:24:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1336 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:24:35 dist-test-slave-pd0t krb5kdc[26913](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:24:35 dist-test-slave-pd0t krb5kdc[26913](info): set up 2 sockets
Sep 26 06:24:35 dist-test-slave-pd0t krb5kdc[26913](info): commencing operation
krb5kdc: starting...
W20260926 06:24:36.716171 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:24:36 dist-test-slave-pd0t krb5kdc[26913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:24:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:24:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.202.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
I20260926 06:24:36.829706 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:40299
--webserver_interface=127.4.202.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:44765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:40299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:24:36.907907 26926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:36.908138 26926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:36.908174 26926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:36.909289 26926 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:24:36.909358 26926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:36.909405 26926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:36.909427 26926 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:24:36.909461 26926 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:24:36.910920 26926 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:44765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:40299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:40299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:36.911180 26926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:36.911398 26926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:36.916134 26932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:36.916157 26934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:36.916221 26931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:36.916373 26926 server_base.cc:1034] running on GCE node
I20260926 06:24:36.916579 26926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:36.916796 26926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:36.917982 26926 hybrid_clock.cc:648] HybridClock initialized: now 1790403876917969 us; error 26 us; skew 500 ppm
Sep 26 06:24:36 dist-test-slave-pd0t krb5kdc[26913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403876, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:36.920378 26926 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:24:36.920806 26926 webserver.cc:460] Webserver started at http://127.4.202.190:39479/ using document root <none> and password file <none>
I20260926 06:24:36.920984 26926 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:36.921022 26926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:36.921108 26926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:36.922034 26926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "fd488e3127984864984c292fc6584570"
format_stamp: "Formatted at 2026-09-26 06:24:36 on dist-test-slave-pd0t"
server_key: "b5bbe47b232de1431a4e388371244d0e"
server_key_iv: "7f28674ea842ff5469572eeb09970cbd"
server_key_version: "encryptionkey@0"
I20260926 06:24:36.922320 26926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "fd488e3127984864984c292fc6584570"
format_stamp: "Formatted at 2026-09-26 06:24:36 on dist-test-slave-pd0t"
server_key: "b5bbe47b232de1431a4e388371244d0e"
server_key_iv: "7f28674ea842ff5469572eeb09970cbd"
server_key_version: "encryptionkey@0"
I20260926 06:24:36.923605 26926 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 06:24:36.924316 26941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:36.924513 26926 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 06:24:36.924584 26926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "fd488e3127984864984c292fc6584570"
format_stamp: "Formatted at 2026-09-26 06:24:36 on dist-test-slave-pd0t"
server_key: "b5bbe47b232de1431a4e388371244d0e"
server_key_iv: "7f28674ea842ff5469572eeb09970cbd"
server_key_version: "encryptionkey@0"
I20260926 06:24:36.924635 26926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:36.945842 26926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:36.948282 26926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:36.948395 26926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:36.952332 26926 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:40299
I20260926 06:24:36.952390 26993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:40299 every 8 connection(s)
I20260926 06:24:36.952744 26926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:24:36.953552 26994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:36.953850 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 26926
I20260926 06:24:36.953984 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:24:36.954181 4906 external_mini_cluster.cc:1428] Setting key 9f91ce510907cb69306412a95b0e6724
I20260926 06:24:36.956949 26994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd488e3127984864984c292fc6584570: Bootstrap starting.
I20260926 06:24:36.957614 26994 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd488e3127984864984c292fc6584570: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:36.957916 26994 log.cc:824] T 00000000000000000000000000000000 P fd488e3127984864984c292fc6584570: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:36.958628 26994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd488e3127984864984c292fc6584570: No bootstrap required, opened a new log
I20260926 06:24:36.959662 26997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:36.954697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58069 (local address 127.4.202.190:40299)
0926 06:24:36.954884 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:36.954891 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:36.954915 (+ 24us) server_negotiation.cc:407] Connection header received
0926 06:24:36.955354 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:36.955360 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:36.955404 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:36.955523 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:36.955755 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:36.956595 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:36.957695 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:36.958169 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:36.958289 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:36.958991 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:36.958995 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:36.958996 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:36.959189 (+ 193us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:36.959230 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:36.959244 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:36.959324 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:36.959396 (+ 72us) server_negotiation.cc:299] Negotiation successful
0926 06:24:36.959456 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":73,"threads_started":1}
I20260926 06:24:36.960728 4906 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:24:36.961190 26994 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd488e3127984864984c292fc6584570 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd488e3127984864984c292fc6584570" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 40299 } }
I20260926 06:24:36.961367 26994 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd488e3127984864984c292fc6584570 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:36.961405 26994 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd488e3127984864984c292fc6584570 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd488e3127984864984c292fc6584570, State: Initialized, Role: FOLLOWER
I20260926 06:24:36.961522 26994 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd488e3127984864984c292fc6584570 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd488e3127984864984c292fc6584570" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 40299 } }
I20260926 06:24:36.961594 26994 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd488e3127984864984c292fc6584570 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:24:36.961623 26994 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd488e3127984864984c292fc6584570 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:24:36.961666 26994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd488e3127984864984c292fc6584570 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:36.962409 26994 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd488e3127984864984c292fc6584570 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd488e3127984864984c292fc6584570" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 40299 } }
I20260926 06:24:36.962520 26994 leader_election.cc:302] T 00000000000000000000000000000000 P fd488e3127984864984c292fc6584570 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd488e3127984864984c292fc6584570; no voters:
I20260926 06:24:36.962708 26994 leader_election.cc:288] T 00000000000000000000000000000000 P fd488e3127984864984c292fc6584570 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:24:36.962795 27004 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd488e3127984864984c292fc6584570 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:24:36.962962 27004 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd488e3127984864984c292fc6584570 [term 1 LEADER]: Becoming Leader. State: Replica: fd488e3127984864984c292fc6584570, State: Running, Role: LEADER
I20260926 06:24:36.962945 26994 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd488e3127984864984c292fc6584570 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:24:36.963136 27004 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd488e3127984864984c292fc6584570 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd488e3127984864984c292fc6584570" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 40299 } }
I20260926 06:24:36.963444 27004 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd488e3127984864984c292fc6584570 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd488e3127984864984c292fc6584570. Latest consensus state: current_term: 1 leader_uuid: "fd488e3127984864984c292fc6584570" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd488e3127984864984c292fc6584570" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 40299 } } }
I20260926 06:24:36.963455 27006 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd488e3127984864984c292fc6584570 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd488e3127984864984c292fc6584570" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd488e3127984864984c292fc6584570" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 40299 } } }
I20260926 06:24:36.963529 27004 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd488e3127984864984c292fc6584570 [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:36.963531 27006 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd488e3127984864984c292fc6584570 [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:36.963742 27010 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:24:36.964299 27010 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:24:36.965924 26997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:36.962203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44737 (local address 127.4.202.190:40299)
0926 06:24:36.962243 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:36.962245 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:36.962382 (+ 137us) server_negotiation.cc:407] Connection header received
0926 06:24:36.962421 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:36.962422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:36.962431 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:36.962501 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:36.962657 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:36.963385 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:36.964561 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:36.964872 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:36.964918 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:36.965623 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:36.965624 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:36.965625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:36.965628 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:36.965641 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:36.965644 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:36.965721 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:36.965803 (+ 82us) server_negotiation.cc:299] Negotiation successful
0926 06:24:36.965824 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 06:24:36.966020 27010 catalog_manager.cc:1347] Generated new cluster ID: c7476da883ef48d4aa96cba637ba8168
I20260926 06:24:36.966064 27010 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:24:36.980405 27010 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:24:36.981086 27010 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:24:36.984508 27010 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd488e3127984864984c292fc6584570: Generated new TSK 0
I20260926 06:24:36.984658 27010 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:24:36.987792 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 26926
2026-09-26T06:24:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1296 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:24:37 dist-test-slave-pd0t krb5kdc[27025](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:24:37 dist-test-slave-pd0t krb5kdc[27025](info): set up 2 sockets
Sep 26 06:24:37 dist-test-slave-pd0t krb5kdc[27025](info): commencing operation
krb5kdc: starting...
W20260926 06:24:38.014783 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:24:38 dist-test-slave-pd0t krb5kdc[27025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:24:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:24:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.202.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
I20260926 06:24:38.131594 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:35681
--webserver_interface=127.4.202.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:45951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:35681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:24:38.210175 27038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:38.210403 27038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:38.210443 27038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:38.211570 27038 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:24:38.211635 27038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:38.211658 27038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:38.211706 27038 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:24:38.211759 27038 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:24:38.213222 27038 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:45951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:35681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:35681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:38.213474 27038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:38.213711 27038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:38.218325 27043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:38.218338 27046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:38.218380 27044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:38.218511 27038 server_base.cc:1034] running on GCE node
I20260926 06:24:38.218781 27038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:38.219045 27038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:38.220188 27038 hybrid_clock.cc:648] HybridClock initialized: now 1790403878220173 us; error 32 us; skew 500 ppm
Sep 26 06:24:38 dist-test-slave-pd0t krb5kdc[27025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403878, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:38.225853 27038 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:24:38.226251 27038 webserver.cc:460] Webserver started at http://127.4.202.190:33789/ using document root <none> and password file <none>
I20260926 06:24:38.226488 27038 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:38.226557 27038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:38.226753 27038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:38.227725 27038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "afa31d1158eb422b8d63e99fb24032a7"
format_stamp: "Formatted at 2026-09-26 06:24:38 on dist-test-slave-pd0t"
server_key: "4c2f24e2bd35b619aab53f58ba746641"
server_key_iv: "8ba75c453d3f66077fa2d7e745d69327"
server_key_version: "encryptionkey@0"
I20260926 06:24:38.228071 27038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "afa31d1158eb422b8d63e99fb24032a7"
format_stamp: "Formatted at 2026-09-26 06:24:38 on dist-test-slave-pd0t"
server_key: "4c2f24e2bd35b619aab53f58ba746641"
server_key_iv: "8ba75c453d3f66077fa2d7e745d69327"
server_key_version: "encryptionkey@0"
I20260926 06:24:38.229339 27038 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:24:38.230063 27053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:38.230329 27038 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:24:38.230409 27038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "afa31d1158eb422b8d63e99fb24032a7"
format_stamp: "Formatted at 2026-09-26 06:24:38 on dist-test-slave-pd0t"
server_key: "4c2f24e2bd35b619aab53f58ba746641"
server_key_iv: "8ba75c453d3f66077fa2d7e745d69327"
server_key_version: "encryptionkey@0"
I20260926 06:24:38.230461 27038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:38.233615 27038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:38.235807 27038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:38.235908 27038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:38.239645 27038 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:35681
I20260926 06:24:38.239714 27105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:35681 every 8 connection(s)
I20260926 06:24:38.240082 27038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:24:38.241014 27106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:38.243219 27106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afa31d1158eb422b8d63e99fb24032a7: Bootstrap starting.
I20260926 06:24:38.243767 27106 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afa31d1158eb422b8d63e99fb24032a7: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:38.244053 27106 log.cc:824] T 00000000000000000000000000000000 P afa31d1158eb422b8d63e99fb24032a7: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:38.244725 27106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afa31d1158eb422b8d63e99fb24032a7: No bootstrap required, opened a new log
I20260926 06:24:38.245576 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 27038
I20260926 06:24:38.245661 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:24:38.245873 4906 external_mini_cluster.cc:1428] Setting key 66050ec8971f9c33809f1572905e4c6b
I20260926 06:24:38.246877 27106 raft_consensus.cc:348] T 00000000000000000000000000000000 P afa31d1158eb422b8d63e99fb24032a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afa31d1158eb422b8d63e99fb24032a7" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 35681 } }
I20260926 06:24:38.247047 27106 raft_consensus.cc:374] T 00000000000000000000000000000000 P afa31d1158eb422b8d63e99fb24032a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 06:24:38.247090 27106 raft_consensus.cc:729] T 00000000000000000000000000000000 P afa31d1158eb422b8d63e99fb24032a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afa31d1158eb422b8d63e99fb24032a7, State: Initialized, Role: FOLLOWER
I20260926 06:24:38.247270 27106 consensus_queue.cc:260] T 00000000000000000000000000000000 P afa31d1158eb422b8d63e99fb24032a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afa31d1158eb422b8d63e99fb24032a7" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 35681 } }
I20260926 06:24:38.247373 27106 raft_consensus.cc:388] T 00000000000000000000000000000000 P afa31d1158eb422b8d63e99fb24032a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 06:24:38.247396 27106 raft_consensus.cc:482] T 00000000000000000000000000000000 P afa31d1158eb422b8d63e99fb24032a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 06:24:38.247489 27106 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afa31d1158eb422b8d63e99fb24032a7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:38.248165 27110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:38.246363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57447 (local address 127.4.202.190:35681)
0926 06:24:38.246524 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:38.246530 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:38.246552 (+ 22us) server_negotiation.cc:407] Connection header received
0926 06:24:38.246862 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:38.246868 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:38.246891 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:38.246999 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:38.247666 (+ 667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:38.247669 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:38.247669 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:38.247672 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:38.247707 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:38.247717 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:38.247806 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:38.247904 (+ 98us) server_negotiation.cc:299] Negotiation successful
0926 06:24:38.247933 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":82,"threads_started":1}
I20260926 06:24:38.248553 27106 raft_consensus.cc:504] T 00000000000000000000000000000000 P afa31d1158eb422b8d63e99fb24032a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afa31d1158eb422b8d63e99fb24032a7" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 35681 } }
I20260926 06:24:38.248653 27106 leader_election.cc:302] T 00000000000000000000000000000000 P afa31d1158eb422b8d63e99fb24032a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: afa31d1158eb422b8d63e99fb24032a7; no voters:
I20260926 06:24:38.248842 27106 leader_election.cc:288] T 00000000000000000000000000000000 P afa31d1158eb422b8d63e99fb24032a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 06:24:38.248860 27111 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afa31d1158eb422b8d63e99fb24032a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:24:38.249073 27111 raft_consensus.cc:686] T 00000000000000000000000000000000 P afa31d1158eb422b8d63e99fb24032a7 [term 1 LEADER]: Becoming Leader. State: Replica: afa31d1158eb422b8d63e99fb24032a7, State: Running, Role: LEADER
I20260926 06:24:38.249107 4906 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:24:38.249126 27106 sys_catalog.cc:564] T 00000000000000000000000000000000 P afa31d1158eb422b8d63e99fb24032a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:24:38.249154 27111 consensus_queue.cc:237] T 00000000000000000000000000000000 P afa31d1158eb422b8d63e99fb24032a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afa31d1158eb422b8d63e99fb24032a7" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 35681 } }
I20260926 06:24:38.249451 27111 sys_catalog.cc:455] T 00000000000000000000000000000000 P afa31d1158eb422b8d63e99fb24032a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "afa31d1158eb422b8d63e99fb24032a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afa31d1158eb422b8d63e99fb24032a7" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 35681 } } }
I20260926 06:24:38.249521 27111 sys_catalog.cc:458] T 00000000000000000000000000000000 P afa31d1158eb422b8d63e99fb24032a7 [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:38.249640 27111 sys_catalog.cc:455] T 00000000000000000000000000000000 P afa31d1158eb422b8d63e99fb24032a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader afa31d1158eb422b8d63e99fb24032a7. Latest consensus state: current_term: 1 leader_uuid: "afa31d1158eb422b8d63e99fb24032a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afa31d1158eb422b8d63e99fb24032a7" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 35681 } } }
I20260926 06:24:38.249697 27111 sys_catalog.cc:458] T 00000000000000000000000000000000 P afa31d1158eb422b8d63e99fb24032a7 [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:38.249821 27117 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:24:38.250484 27117 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:24:38.252116 27110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:38.250902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36021 (local address 127.4.202.190:35681)
0926 06:24:38.250954 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:38.250956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:38.251145 (+ 189us) server_negotiation.cc:407] Connection header received
0926 06:24:38.251200 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:38.251201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:38.251207 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:38.251276 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:38.251920 (+ 644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:38.251921 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:38.251921 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 06:24:38.251925 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:38.251937 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 06:24:38.251941 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:38.251987 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:38.252041 (+ 54us) server_negotiation.cc:299] Negotiation successful
0926 06:24:38.252055 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 06:24:38.252377 27117 catalog_manager.cc:1347] Generated new cluster ID: 11a5b9c90c774c53904bb649fe31c669
I20260926 06:24:38.252452 27117 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:24:38.265964 27117 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:24:38.266105 27117 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:24:38.276244 27117 catalog_manager.cc:5868] T 00000000000000000000000000000000 P afa31d1158eb422b8d63e99fb24032a7: Generated new TSK 0
I20260926 06:24:38.276434 27117 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:24:38.285544 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 27038
2026-09-26T06:24:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1299 ms)
[----------] 18 tests from AuthTokenIssuingTest (23328 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:24:38 dist-test-slave-pd0t krb5kdc[27137](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:24:38 dist-test-slave-pd0t krb5kdc[27137](info): set up 2 sockets
Sep 26 06:24:38 dist-test-slave-pd0t krb5kdc[27137](info): commencing operation
krb5kdc: starting...
W20260926 06:24:39.314750 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:24:39 dist-test-slave-pd0t krb5kdc[27137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:24:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:24:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.202.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
I20260926 06:24:39.428831 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:45711
--webserver_interface=127.4.202.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:44105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:45711,127.4.202.189:37733,127.4.202.188:34563
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:24:39.509518 27150 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 06:24:39.509706 27150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:39.509722 27150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:39.509742 27150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:39.510823 27150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:24:39.510856 27150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:39.510870 27150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:39.510882 27150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:24:39.510895 27150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:24:39.512281 27150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:44105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:45711,127.4.202.189:37733,127.4.202.188:34563
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:45711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:39.512457 27150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:39.512691 27150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:39.517345 27155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:39.517381 27156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:39.517535 27150 server_base.cc:1034] running on GCE node
W20260926 06:24:39.517381 27158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:39.517853 27150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:39.518146 27150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:39.519289 27150 hybrid_clock.cc:648] HybridClock initialized: now 1790403879519269 us; error 39 us; skew 500 ppm
Sep 26 06:24:39 dist-test-slave-pd0t krb5kdc[27137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403879, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:39.521708 27150 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:24:39.522132 27150 webserver.cc:460] Webserver started at http://127.4.202.190:36625/ using document root <none> and password file <none>
I20260926 06:24:39.522368 27150 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:39.522413 27150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:39.522619 27150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:39.523636 27150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "79be6c789b4b434daa0addb84454e41a"
format_stamp: "Formatted at 2026-09-26 06:24:39 on dist-test-slave-pd0t"
server_key: "640eb64e388818bb03b546abf01c4022"
server_key_iv: "84dbb67b8012323afa1335fc162d0426"
server_key_version: "encryptionkey@0"
I20260926 06:24:39.523990 27150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "79be6c789b4b434daa0addb84454e41a"
format_stamp: "Formatted at 2026-09-26 06:24:39 on dist-test-slave-pd0t"
server_key: "640eb64e388818bb03b546abf01c4022"
server_key_iv: "84dbb67b8012323afa1335fc162d0426"
server_key_version: "encryptionkey@0"
I20260926 06:24:39.525229 27150 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:24:39.525928 27165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:39.526118 27150 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:24:39.526194 27150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "79be6c789b4b434daa0addb84454e41a"
format_stamp: "Formatted at 2026-09-26 06:24:39 on dist-test-slave-pd0t"
server_key: "640eb64e388818bb03b546abf01c4022"
server_key_iv: "84dbb67b8012323afa1335fc162d0426"
server_key_version: "encryptionkey@0"
I20260926 06:24:39.526244 27150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:39.566175 27150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:39.568430 27150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:39.568534 27150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:39.572332 27150 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:45711
I20260926 06:24:39.572407 27217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:45711 every 8 connection(s)
I20260926 06:24:39.572728 27150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:24:39.573390 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 27150
I20260926 06:24:39.573467 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:24:39.573680 4906 external_mini_cluster.cc:1428] Setting key 4e249c6412a23291299f6c81da366a08
I20260926 06:24:39.573839 27218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:39.575816 27218 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45711 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.4.202.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.189@KRBTEST.COM" created.
Sep 26 06:24:39 dist-test-slave-pd0t krb5kdc[27137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403879, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:24:39.587968 27221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.576627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:45711 (local address 127.0.0.1:42672)
0926 06:24:39.576812 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:39.576852 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:39.577325 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:39.577619 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:39.577621 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:39.577895 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:39.578039 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:39.578043 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.579190 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.579191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:39.579952 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:39.579954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.580544 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.580545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:39.580608 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:39.581122 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:24:39.581130 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:39.584535 (+ 3405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:39.586410 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:39.586416 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:39.586418 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:39.586657 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:39.586925 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:39.586926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:39.586927 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:39.587018 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:39.587366 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:39.587371 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:39.587433 (+ 62us) client_negotiation.cc:769] Sending connection context
0926 06:24:39.587544 (+ 111us) client_negotiation.cc:240] Negotiation successful
0926 06:24:39.587777 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":101,"thread_start_us":96,"threads_started":1}
I20260926 06:24:39.587975 27222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.576657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42672 (local address 127.4.202.190:45711)
0926 06:24:39.576896 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:39.576901 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:39.576934 (+ 33us) server_negotiation.cc:407] Connection header received
0926 06:24:39.577484 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:39.577486 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:39.577506 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:39.577615 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:39.578150 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.579080 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.580029 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.580425 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.580564 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:39.584625 (+ 4061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:39.584633 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:39.584633 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:39.584641 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:39.586305 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:39.586761 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:39.586764 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:39.586764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:39.586851 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:39.587087 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:39.587088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:39.587088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:39.587288 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:39.587387 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:39.587576 (+ 189us) server_negotiation.cc:299] Negotiation successful
0926 06:24:39.587716 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":97,"threads_started":1}
I20260926 06:24:39.588690 27218 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.202.189" port: 37733 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:24:39.589074 27221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.588941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:37733 (local address 127.0.0.1:47158)
0926 06:24:39.588999 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:39.589049 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.189:37733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260926 06:24:39.589231 27167 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.202.189:37733: connect: Connection refused (error 111)
I20260926 06:24:39.589679 27221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.589554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:37733 (local address 127.0.0.1:47162)
0926 06:24:39.589616 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:39.589647 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.189:37733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260926 06:24:39.589823 27218 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.202.189:37733: Network error: Client connection negotiation failed: client connection to 127.4.202.189:37733: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.189.keytab.
Entry for principal kudu/127.4.202.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.189.keytab.
I20260926 06:24:39.598853 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.189:37733
--webserver_interface=127.4.202.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.189.keytab
--principal=kudu/127.4.202.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:44105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:45711,127.4.202.189:37733,127.4.202.188:34563
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:24:39.639051 27218 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.202.189" port: 37733 } attempt: 1
I20260926 06:24:39.639508 27221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.639368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:37733 (local address 127.0.0.1:47168)
0926 06:24:39.639434 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:39.639475 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.189:37733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260926 06:24:39.639942 27221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.639846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:37733 (local address 127.0.0.1:47170)
0926 06:24:39.639896 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:39.639922 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.189:37733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260926 06:24:39.640046 27218 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.202.189:37733: Network error: Client connection negotiation failed: client connection to 127.4.202.189:37733: connect: Connection refused (error 111)
W20260926 06:24:39.678140 27225 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 06:24:39.678332 27225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:39.678349 27225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:39.678368 27225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:39.679430 27225 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:24:39.679462 27225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:39.679476 27225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:39.679488 27225 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:24:39.679500 27225 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:24:39.681052 27225 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:44105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:45711,127.4.202.189:37733,127.4.202.188:34563
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.189.keytab
--principal=kudu/127.4.202.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.189:37733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.202.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:39.681259 27225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:39.681417 27225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:39.686118 27230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:39.686118 27233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:39.686157 27231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:39.686305 27225 server_base.cc:1034] running on GCE node
I20260926 06:24:39.686699 27225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:39.686992 27225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:39.688164 27225 hybrid_clock.cc:648] HybridClock initialized: now 1790403879688151 us; error 25 us; skew 500 ppm
Sep 26 06:24:39 dist-test-slave-pd0t krb5kdc[27137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403879, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:39.690500 27225 init.cc:377] Logged in from keytab as kudu/127.4.202.189@KRBTEST.COM (short username kudu)
I20260926 06:24:39.690944 27225 webserver.cc:460] Webserver started at http://127.4.202.189:46415/ using document root <none> and password file <none>
I20260926 06:24:39.691169 27225 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:39.691241 27225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:39.691393 27225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:39.692404 27225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-1/data/instance:
uuid: "6fd5746b66294aba81fddccef17a0803"
format_stamp: "Formatted at 2026-09-26 06:24:39 on dist-test-slave-pd0t"
server_key: "a740741ab02c57e8e9f77f93e9a45939"
server_key_iv: "ef92adecdfa58ecc81b0f7b6657a026f"
server_key_version: "encryptionkey@0"
I20260926 06:24:39.692766 27225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-1/wal/instance:
uuid: "6fd5746b66294aba81fddccef17a0803"
format_stamp: "Formatted at 2026-09-26 06:24:39 on dist-test-slave-pd0t"
server_key: "a740741ab02c57e8e9f77f93e9a45939"
server_key_iv: "ef92adecdfa58ecc81b0f7b6657a026f"
server_key_version: "encryptionkey@0"
I20260926 06:24:39.694023 27225 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 06:24:39.694768 27240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:39.694959 27225 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 06:24:39.695060 27225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-1/wal
uuid: "6fd5746b66294aba81fddccef17a0803"
format_stamp: "Formatted at 2026-09-26 06:24:39 on dist-test-slave-pd0t"
server_key: "a740741ab02c57e8e9f77f93e9a45939"
server_key_iv: "ef92adecdfa58ecc81b0f7b6657a026f"
server_key_version: "encryptionkey@0"
I20260926 06:24:39.695142 27225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:39.704349 27225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:39.706008 27225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:39.706174 27225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:39.708204 27218 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.202.189" port: 37733 } attempt: 2
I20260926 06:24:39.708635 27221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.708487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:37733 (local address 127.0.0.1:47176)
0926 06:24:39.708555 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:39.708592 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.189:37733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260926 06:24:39.709049 27221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.708909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:37733 (local address 127.0.0.1:47188)
0926 06:24:39.708974 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:39.709010 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.189:37733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260926 06:24:39.709151 27218 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.202.189:37733: Network error: Client connection negotiation failed: client connection to 127.4.202.189:37733: connect: Connection refused (error 111)
I20260926 06:24:39.710328 27225 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.189:37733
I20260926 06:24:39.710383 27292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.189:37733 every 8 connection(s)
I20260926 06:24:39.710749 27225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-1/data/info.pb
I20260926 06:24:39.711710 27293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:39.713017 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 27225
I20260926 06:24:39.713133 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-1/wal/instance
I20260926 06:24:39.713325 4906 external_mini_cluster.cc:1428] Setting key 8d6a5e309a067dc2c3dd55b9c38e7313
I20260926 06:24:39.713356 27293 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45711 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 06:24:39 dist-test-slave-pd0t krb5kdc[27137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403879, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.189@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.202.188@KRBTEST.COM; defaulting to no policy
I20260926 06:24:39.722924 27222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.714080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42676 (local address 127.4.202.190:45711)
0926 06:24:39.714141 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:39.714143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:39.714402 (+ 259us) server_negotiation.cc:407] Connection header received
0926 06:24:39.715099 (+ 697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:39.715101 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:39.715115 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:39.715190 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:39.715725 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.716506 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.717552 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.717928 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.718022 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:39.720082 (+ 2060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:39.720083 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:39.720084 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:39.720092 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:39.721574 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:39.721935 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:39.721936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:39.721936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:39.721984 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:39.722334 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:39.722335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:39.722335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:39.722434 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:39.722488 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:39.722803 (+ 315us) server_negotiation.cc:299] Negotiation successful
0926 06:24:39.722867 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 06:24:39.723124 27295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.714073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:45711 (local address 127.0.0.1:42676)
0926 06:24:39.714266 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:39.714310 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:39.715002 (+ 692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:39.715238 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:39.715241 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:39.715466 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:39.715639 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:39.715643 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.716599 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.716600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:39.717460 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:39.717463 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.718040 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.718041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:39.718138 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:39.718693 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:24:39.718701 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:39.719985 (+ 1284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:39.721650 (+ 1665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:39.721656 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:39.721658 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:39.721868 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:39.722068 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:39.722069 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:39.722070 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:39.722263 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:39.722513 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:39.722519 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:39.722687 (+ 168us) client_negotiation.cc:769] Sending connection context
0926 06:24:39.722776 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 06:24:39.722883 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":100,"threads_started":1}
I20260926 06:24:39.723719 27293 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.202.189" port: 37733 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.188@KRBTEST.COM" created.
Sep 26 06:24:39 dist-test-slave-pd0t krb5kdc[27137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403879, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.189@KRBTEST.COM for kudu/127.4.202.189@KRBTEST.COM
I20260926 06:24:39.732108 27295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.724044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:37733 (local address 127.0.0.1:47202)
0926 06:24:39.724096 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:39.724108 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:39.724151 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:39.724416 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:39.724417 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:39.724524 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:39.724570 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:39.724571 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.725772 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.725774 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:39.726662 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:39.726664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.727139 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.727141 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:39.727225 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:39.727613 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:24:39.727619 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:39.729063 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:39.731085 (+ 2022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:39.731087 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:39.731087 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:39.731282 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:39.731537 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:39.731537 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:39.731538 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:39.731606 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:39.731888 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:39.731889 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:39.731923 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 06:24:39.731980 (+ 57us) client_negotiation.cc:240] Negotiation successful
0926 06:24:39.732052 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 06:24:39.732265 27297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.724106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47202 (local address 127.4.202.189:37733)
0926 06:24:39.724258 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:39.724264 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:39.724286 (+ 22us) server_negotiation.cc:407] Connection header received
0926 06:24:39.724312 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:39.724314 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:39.724327 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:39.724411 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:39.724650 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.725669 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.726761 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.727054 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.727113 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:39.729171 (+ 2058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:39.729189 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:39.729190 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:39.729196 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:39.731003 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:39.731386 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:39.731388 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:39.731389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:39.731459 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:39.731673 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:39.731673 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:39.731674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:39.731820 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:39.731882 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:39.732000 (+ 118us) server_negotiation.cc:299] Negotiation successful
0926 06:24:39.732099 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":127,"thread_start_us":72,"threads_started":1}
I20260926 06:24:39.732853 27293 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.202.188" port: 34563 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:24:39.733142 27295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.733062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:34563 (local address 127.0.0.1:51844)
0926 06:24:39.733088 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:39.733125 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.188:34563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260926 06:24:39.733306 27243 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.202.188:34563: connect: Connection refused (error 111)
I20260926 06:24:39.733711 27295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.733615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:34563 (local address 127.0.0.1:51846)
0926 06:24:39.733665 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:39.733690 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.188:34563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260926 06:24:39.733860 27293 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.202.188:34563: Network error: Client connection negotiation failed: client connection to 127.4.202.188:34563: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.188.keytab.
Entry for principal kudu/127.4.202.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.188.keytab.
I20260926 06:24:39.739359 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.188:34563
--webserver_interface=127.4.202.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.188.keytab
--principal=kudu/127.4.202.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:44105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:45711,127.4.202.189:37733,127.4.202.188:34563
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:24:39.783105 27293 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.202.188" port: 34563 } attempt: 1
I20260926 06:24:39.783620 27295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.783488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:34563 (local address 127.0.0.1:51854)
0926 06:24:39.783549 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:39.783589 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.188:34563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260926 06:24:39.784039 27295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.783952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:34563 (local address 127.0.0.1:51868)
0926 06:24:39.783998 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:39.784022 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.188:34563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260926 06:24:39.784150 27293 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.202.188:34563: Network error: Client connection negotiation failed: client connection to 127.4.202.188:34563: connect: Connection refused (error 111)
I20260926 06:24:39.800302 27218 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.202.189" port: 37733 } attempt: 3
Sep 26 06:24:39 dist-test-slave-pd0t krb5kdc[27137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403879, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for kudu/127.4.202.189@KRBTEST.COM
I20260926 06:24:39.807667 27297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.800700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47216 (local address 127.4.202.189:37733)
0926 06:24:39.800758 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:39.800760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:39.800858 (+ 98us) server_negotiation.cc:407] Connection header received
0926 06:24:39.800928 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:39.800929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:39.800943 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:39.801026 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:39.801373 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.801853 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.802630 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.802850 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.802938 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:39.804895 (+ 1957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:39.804897 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:39.804897 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:39.804905 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:39.806384 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:39.806890 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:39.806891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:39.806892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:39.806954 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:39.807161 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:39.807162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:39.807163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:39.807275 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:39.807357 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:39.807508 (+ 151us) server_negotiation.cc:299] Negotiation successful
0926 06:24:39.807577 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 06:24:39.807669 27221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.800647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:37733 (local address 127.0.0.1:47216)
0926 06:24:39.800703 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:39.800711 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:39.800862 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:39.801031 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:39.801032 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:39.801203 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:39.801270 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:39.801271 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.801940 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.801941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:39.802480 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:39.802482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.802938 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.802939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:39.803024 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:39.803558 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:24:39.803563 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:39.804802 (+ 1239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:39.806498 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:39.806498 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:39.806499 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:39.806744 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:39.807026 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:39.807027 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:39.807028 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:39.807085 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:39.807347 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:39.807348 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:39.807393 (+ 45us) client_negotiation.cc:769] Sending connection context
0926 06:24:39.807484 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 06:24:39.807577 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260926 06:24:39.808005 27218 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.202.188" port: 34563 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:24:39.808341 27221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.808242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:34563 (local address 127.0.0.1:51876)
0926 06:24:39.808293 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:39.808318 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.188:34563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260926 06:24:39.808749 27221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.808663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:34563 (local address 127.0.0.1:51878)
0926 06:24:39.808712 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:39.808735 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.188:34563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260926 06:24:39.808866 27218 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.202.188:34563: Network error: Client connection negotiation failed: client connection to 127.4.202.188:34563: connect: Connection refused (error 111)
W20260926 06:24:39.820706 27300 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 06:24:39.820896 27300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:39.820914 27300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:39.820966 27300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:39.822100 27300 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:24:39.822140 27300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:39.822154 27300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:39.822206 27300 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:24:39.822227 27300 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:24:39.823858 27300 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:44105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:45711,127.4.202.189:37733,127.4.202.188:34563
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.188.keytab
--principal=kudu/127.4.202.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.188:34563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.202.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:39.824106 27300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:39.824333 27300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:39.828936 27308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:39.828992 27305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:39.828972 27306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:39.829293 27300 server_base.cc:1034] running on GCE node
I20260926 06:24:39.829561 27300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:39.829888 27300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:39.831061 27300 hybrid_clock.cc:648] HybridClock initialized: now 1790403879831011 us; error 45 us; skew 500 ppm
Sep 26 06:24:39 dist-test-slave-pd0t krb5kdc[27137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403879, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:39.833454 27300 init.cc:377] Logged in from keytab as kudu/127.4.202.188@KRBTEST.COM (short username kudu)
I20260926 06:24:39.833825 27300 webserver.cc:460] Webserver started at http://127.4.202.188:32897/ using document root <none> and password file <none>
I20260926 06:24:39.834079 27300 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:39.834151 27300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:39.834301 27300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:39.835394 27300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-2/data/instance:
uuid: "2379fdf9c84d4730b6513cb71d43c9af"
format_stamp: "Formatted at 2026-09-26 06:24:39 on dist-test-slave-pd0t"
server_key: "5afd568c2909a6a2f27f426b7a5678f4"
server_key_iv: "4d083c320016af65d3babaa4cdcfd4df"
server_key_version: "encryptionkey@0"
I20260926 06:24:39.835763 27300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-2/wal/instance:
uuid: "2379fdf9c84d4730b6513cb71d43c9af"
format_stamp: "Formatted at 2026-09-26 06:24:39 on dist-test-slave-pd0t"
server_key: "5afd568c2909a6a2f27f426b7a5678f4"
server_key_iv: "4d083c320016af65d3babaa4cdcfd4df"
server_key_version: "encryptionkey@0"
I20260926 06:24:39.837036 27300 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 06:24:39.837703 27315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:39.837890 27300 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 06:24:39.837982 27300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-2/wal
uuid: "2379fdf9c84d4730b6513cb71d43c9af"
format_stamp: "Formatted at 2026-09-26 06:24:39 on dist-test-slave-pd0t"
server_key: "5afd568c2909a6a2f27f426b7a5678f4"
server_key_iv: "4d083c320016af65d3babaa4cdcfd4df"
server_key_version: "encryptionkey@0"
I20260926 06:24:39.838064 27300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:39.839996 27218 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.202.188" port: 34563 } attempt: 1
I20260926 06:24:39.840360 27221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.840235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:34563 (local address 127.0.0.1:51880)
0926 06:24:39.840299 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:39.840335 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.188:34563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260926 06:24:39.840670 27221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.840580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:34563 (local address 127.0.0.1:51884)
0926 06:24:39.840623 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:39.840646 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.188:34563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260926 06:24:39.840834 27218 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.202.188:34563: Network error: Client connection negotiation failed: client connection to 127.4.202.188:34563: connect: Connection refused (error 111)
I20260926 06:24:39.848034 27300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:39.849685 27300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:39.849848 27300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:39.852438 27293 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.202.188" port: 34563 } attempt: 2
I20260926 06:24:39.852792 27295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.852674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:34563 (local address 127.0.0.1:51890)
0926 06:24:39.852734 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:39.852762 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.188:34563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260926 06:24:39.853142 27295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.853067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:34563 (local address 127.0.0.1:51896)
0926 06:24:39.853095 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:39.853118 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.188:34563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260926 06:24:39.853276 27293 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.202.188:34563: Network error: Client connection negotiation failed: client connection to 127.4.202.188:34563: connect: Connection refused (error 111)
I20260926 06:24:39.854221 27300 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.188:34563
I20260926 06:24:39.854249 27367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.188:34563 every 8 connection(s)
I20260926 06:24:39.854621 27300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-2/data/info.pb
I20260926 06:24:39.855515 27368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:39.856920 27368 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45711 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 06:24:39 dist-test-slave-pd0t krb5kdc[27137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403879, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.188@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:24:39.863468 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 27300
I20260926 06:24:39.863590 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790403591640726-4906-0/minicluster-data/master-2/wal/instance
I20260926 06:24:39.863790 4906 external_mini_cluster.cc:1428] Setting key 70d77ca603238c88d8556841507c52de
I20260926 06:24:39.865686 27222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.857492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42690 (local address 127.4.202.190:45711)
0926 06:24:39.857550 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:39.857551 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:39.857726 (+ 175us) server_negotiation.cc:407] Connection header received
0926 06:24:39.858145 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:39.858146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:39.858154 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:39.858206 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:39.858850 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.859305 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.860232 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.860415 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.860468 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:39.862447 (+ 1979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:39.862448 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:39.862449 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:39.862455 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:39.864062 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:39.864707 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:39.864709 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:39.864710 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:39.864772 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:39.865025 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:39.865025 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:39.865026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:39.865168 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:39.865245 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:39.865571 (+ 326us) server_negotiation.cc:299] Negotiation successful
0926 06:24:39.865630 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 06:24:39.865877 27370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.857440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:45711 (local address 127.0.0.1:42690)
0926 06:24:39.857603 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:39.857640 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:39.858068 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:39.858261 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:39.858264 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:39.858511 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:39.858771 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:39.858776 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.859384 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.859385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:39.860158 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:39.860160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.860499 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.860499 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:39.860599 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:39.861021 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:24:39.861029 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:39.862362 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:39.864324 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:39.864327 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:39.864329 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:39.864624 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:39.864860 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:39.864860 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:39.864861 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:39.864954 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:39.865277 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:39.865282 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:39.865427 (+ 145us) client_negotiation.cc:769] Sending connection context
0926 06:24:39.865544 (+ 117us) client_negotiation.cc:240] Negotiation successful
0926 06:24:39.865685 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":88,"threads_started":1}
I20260926 06:24:39.866437 27368 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.202.189" port: 37733 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 06:24:39 dist-test-slave-pd0t krb5kdc[27137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.188@KRBTEST.COM
Sep 26 06:24:39 dist-test-slave-pd0t krb5kdc[27137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403879, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.188@KRBTEST.COM for kudu/127.4.202.189@KRBTEST.COM
I20260926 06:24:39.872968 27372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.864182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51898 (local address 127.4.202.188:34563)
0926 06:24:39.864330 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:39.864335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:39.864404 (+ 69us) server_negotiation.cc:407] Connection header received
0926 06:24:39.864448 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:39.864451 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:39.864464 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:39.864542 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:39.865275 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.866400 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.867270 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.867573 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.867667 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:39.870171 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:39.870179 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:39.870179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:39.870186 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:39.871685 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:39.872103 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:39.872106 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:39.872106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:39.872177 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:39.872411 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:39.872411 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:39.872412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:39.872547 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:39.872704 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:39.872784 (+ 80us) server_negotiation.cc:299] Negotiation successful
0926 06:24:39.872863 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":99,"threads_started":1}
I20260926 06:24:39.873210 27297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.866786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47224 (local address 127.4.202.189:37733)
0926 06:24:39.866862 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:39.866864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:39.866874 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:24:39.866920 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:39.866921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:39.866930 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:39.866978 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:39.867230 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.867680 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.868330 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.868549 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.868637 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:39.870514 (+ 1877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:39.870515 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:39.870516 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:39.870523 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:39.872021 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:39.872408 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:39.872409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:39.872410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:39.872464 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:39.872671 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:39.872672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:39.872672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:39.872781 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:39.872882 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:39.873043 (+ 161us) server_negotiation.cc:299] Negotiation successful
0926 06:24:39.873121 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 06:24:39.873472 27370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.866714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:37733 (local address 127.0.0.1:47224)
0926 06:24:39.866764 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:39.866776 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:39.866832 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:39.866982 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:39.866982 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:39.867107 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:39.867150 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:39.867152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.867764 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.867765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:39.868244 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:39.868245 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.868638 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.868639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:39.868712 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:39.869105 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:24:39.869110 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:39.870453 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:39.872130 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:39.872131 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:39.872131 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:39.872314 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:39.872539 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:39.872540 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:39.872540 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:39.872597 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:39.872875 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:39.872876 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:39.872917 (+ 41us) client_negotiation.cc:769] Sending connection context
0926 06:24:39.873284 (+ 367us) client_negotiation.cc:240] Negotiation successful
0926 06:24:39.873386 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260926 06:24:39.873813 27368 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.202.188" port: 34563 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 06:24:39 dist-test-slave-pd0t krb5kdc[27137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403879, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.188@KRBTEST.COM for kudu/127.4.202.188@KRBTEST.COM
Sep 26 06:24:39 dist-test-slave-pd0t krb5kdc[27137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:24:39.880151 27370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.874053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:34563 (local address 127.0.0.1:51902)
0926 06:24:39.874089 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:39.874097 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:39.874129 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:39.874259 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:39.874259 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:39.874364 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:39.874388 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:39.874389 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.875037 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.875038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:39.875538 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:39.875540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.875960 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.875961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:39.876020 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:39.876504 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:24:39.876510 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:39.877608 (+ 1098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:39.879174 (+ 1566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:39.879174 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:39.879175 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:39.879342 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:39.879549 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:39.879550 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:39.879551 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:39.879603 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:39.879852 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:39.879852 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:39.879889 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 06:24:39.879956 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 06:24:39.880048 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260926 06:24:39.880420 27372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.874115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51902 (local address 127.4.202.188:34563)
0926 06:24:39.874153 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:39.874154 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:39.874162 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:24:39.874181 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:39.874182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:39.874190 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:39.874258 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:39.874457 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.874968 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.875615 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.875893 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.875941 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:39.877685 (+ 1744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:39.877686 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:39.877687 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:39.877692 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:39.879075 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:39.879428 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:39.879429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:39.879429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:39.879476 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:39.879675 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:39.879675 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:39.879676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:39.879780 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:39.879860 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:39.879988 (+ 128us) server_negotiation.cc:299] Negotiation successful
0926 06:24:39.880094 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":23}
I20260926 06:24:39.881287 27222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.873811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42698 (local address 127.4.202.190:45711)
0926 06:24:39.873845 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:39.873847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:39.873854 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:24:39.873873 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:39.873873 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:39.873882 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:39.873941 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:39.874540 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.875061 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.875740 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.875963 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.876023 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:39.878424 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:39.878425 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:39.878425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:39.878432 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:39.879819 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:39.880198 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:39.880199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:39.880200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:39.880258 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:39.880545 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:39.880546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:39.880547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:39.880724 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:39.880902 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:39.881100 (+ 198us) server_negotiation.cc:299] Negotiation successful
0926 06:24:39.881172 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 06:24:39.882100 27368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2379fdf9c84d4730b6513cb71d43c9af: Bootstrap starting.
I20260926 06:24:39.883046 27368 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2379fdf9c84d4730b6513cb71d43c9af: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:39.883324 27368 log.cc:824] T 00000000000000000000000000000000 P 2379fdf9c84d4730b6513cb71d43c9af: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:39.884053 27368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2379fdf9c84d4730b6513cb71d43c9af: No bootstrap required, opened a new log
I20260926 06:24:39.885502 27368 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2379fdf9c84d4730b6513cb71d43c9af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79be6c789b4b434daa0addb84454e41a" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45711 } } peers { permanent_uuid: "6fd5746b66294aba81fddccef17a0803" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 37733 } } peers { permanent_uuid: "2379fdf9c84d4730b6513cb71d43c9af" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 34563 } }
I20260926 06:24:39.885632 27368 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2379fdf9c84d4730b6513cb71d43c9af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2379fdf9c84d4730b6513cb71d43c9af, State: Initialized, Role: FOLLOWER
I20260926 06:24:39.885720 27368 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2379fdf9c84d4730b6513cb71d43c9af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79be6c789b4b434daa0addb84454e41a" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45711 } } peers { permanent_uuid: "6fd5746b66294aba81fddccef17a0803" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 37733 } } peers { permanent_uuid: "2379fdf9c84d4730b6513cb71d43c9af" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 34563 } }
I20260926 06:24:39.886003 27374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2379fdf9c84d4730b6513cb71d43c9af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79be6c789b4b434daa0addb84454e41a" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45711 } } peers { permanent_uuid: "6fd5746b66294aba81fddccef17a0803" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 37733 } } peers { permanent_uuid: "2379fdf9c84d4730b6513cb71d43c9af" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 34563 } } }
I20260926 06:24:39.886042 27368 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2379fdf9c84d4730b6513cb71d43c9af [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:24:39.886116 27374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2379fdf9c84d4730b6513cb71d43c9af [sys.catalog]: This master's current role is: FOLLOWER
Sep 26 06:24:39 dist-test-slave-pd0t krb5kdc[27137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.189@KRBTEST.COM
W20260926 06:24:39.888006 27385 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2379fdf9c84d4730b6513cb71d43c9af: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:24:39.888108 27385 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:24:39.890084 27297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.881859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47240 (local address 127.4.202.189:37733)
0926 06:24:39.881892 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:39.881893 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:39.881938 (+ 45us) server_negotiation.cc:407] Connection header received
0926 06:24:39.882005 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:39.882006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:39.882015 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:39.882066 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:39.882646 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.883288 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.884111 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.884393 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.884457 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:39.886775 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:39.886777 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:39.886777 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:39.886784 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:39.888578 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:39.889051 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:39.889052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:39.889053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:39.889110 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:39.889306 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:39.889307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:39.889307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:39.889450 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:39.889583 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:39.889929 (+ 346us) server_negotiation.cc:299] Negotiation successful
0926 06:24:39.889998 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 06:24:39.889032 27300 master_runner.cc:186] Error getting master registration for 127.4.202.190:45711: OK, instance_id { permanent_uuid: "79be6c789b4b434daa0addb84454e41a" instance_seqno: 1790403879571185 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 06:24:39.891223 27300 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 06:24:39.892664 4906 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:24:39.893746 27192 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "79be6c789b4b434daa0addb84454e41a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42698
I20260926 06:24:39.915995 27218 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.202.188" port: 34563 } attempt: 2
Sep 26 06:24:39 dist-test-slave-pd0t krb5kdc[27137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403879, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for kudu/127.4.202.188@KRBTEST.COM
I20260926 06:24:39.923941 27372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.916355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51918 (local address 127.4.202.188:34563)
0926 06:24:39.916397 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:39.916399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:39.916408 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:24:39.916451 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:39.916452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:39.916459 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:39.916530 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:39.916897 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.917641 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.918671 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.918959 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.919021 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:39.921221 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:39.921223 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:39.921223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:39.921229 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:39.922814 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:39.923141 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:39.923142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:39.923142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:39.923200 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:39.923416 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:39.923417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:39.923417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:39.923553 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:39.923631 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:39.923782 (+ 151us) server_negotiation.cc:299] Negotiation successful
0926 06:24:39.923853 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 06:24:39.923941 27221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.916258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:34563 (local address 127.0.0.1:51918)
0926 06:24:39.916312 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:39.916322 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:39.916359 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:39.916568 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:39.916569 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:39.916754 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:39.916805 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:39.916807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.917727 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.917728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:39.918538 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:39.918540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.919037 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.919038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:39.919097 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:39.919654 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:24:39.919659 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:39.921101 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:39.922888 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:39.922889 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:39.922889 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:39.923058 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:39.923289 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:39.923289 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:39.923290 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:39.923347 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:39.923624 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:39.923625 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:39.923669 (+ 44us) client_negotiation.cc:769] Sending connection context
0926 06:24:39.923759 (+ 90us) client_negotiation.cc:240] Negotiation successful
0926 06:24:39.923853 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260926 06:24:39.925720 27218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79be6c789b4b434daa0addb84454e41a: Bootstrap starting.
I20260926 06:24:39.926436 27218 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79be6c789b4b434daa0addb84454e41a: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:39.926749 27218 log.cc:824] T 00000000000000000000000000000000 P 79be6c789b4b434daa0addb84454e41a: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:39.927541 27218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79be6c789b4b434daa0addb84454e41a: No bootstrap required, opened a new log
I20260926 06:24:39.928961 27218 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79be6c789b4b434daa0addb84454e41a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79be6c789b4b434daa0addb84454e41a" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45711 } } peers { permanent_uuid: "6fd5746b66294aba81fddccef17a0803" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 37733 } } peers { permanent_uuid: "2379fdf9c84d4730b6513cb71d43c9af" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 34563 } }
I20260926 06:24:39.929107 27218 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79be6c789b4b434daa0addb84454e41a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79be6c789b4b434daa0addb84454e41a, State: Initialized, Role: FOLLOWER
I20260926 06:24:39.929190 27218 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79be6c789b4b434daa0addb84454e41a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79be6c789b4b434daa0addb84454e41a" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45711 } } peers { permanent_uuid: "6fd5746b66294aba81fddccef17a0803" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 37733 } } peers { permanent_uuid: "2379fdf9c84d4730b6513cb71d43c9af" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 34563 } }
I20260926 06:24:39.929481 27218 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79be6c789b4b434daa0addb84454e41a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:24:39.929451 27388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79be6c789b4b434daa0addb84454e41a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79be6c789b4b434daa0addb84454e41a" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45711 } } peers { permanent_uuid: "6fd5746b66294aba81fddccef17a0803" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 37733 } } peers { permanent_uuid: "2379fdf9c84d4730b6513cb71d43c9af" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 34563 } } }
I20260926 06:24:39.929539 27388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79be6c789b4b434daa0addb84454e41a [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:24:39.929615 27192 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79be6c789b4b434daa0addb84454e41a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260926 06:24:39.929678 27192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79be6c789b4b434daa0addb84454e41a [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:39.930420 27192 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79be6c789b4b434daa0addb84454e41a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79be6c789b4b434daa0addb84454e41a" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45711 } } peers { permanent_uuid: "6fd5746b66294aba81fddccef17a0803" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 37733 } } peers { permanent_uuid: "2379fdf9c84d4730b6513cb71d43c9af" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 34563 } }
I20260926 06:24:39.930955 27342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "79be6c789b4b434daa0addb84454e41a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2379fdf9c84d4730b6513cb71d43c9af"
I20260926 06:24:39.931077 27342 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2379fdf9c84d4730b6513cb71d43c9af [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:39.932053 27342 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2379fdf9c84d4730b6513cb71d43c9af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79be6c789b4b434daa0addb84454e41a in term 1.
I20260926 06:24:39.932221 27192 leader_election.cc:288] T 00000000000000000000000000000000 P 79be6c789b4b434daa0addb84454e41a [CANDIDATE]: Term 1 election: Requested vote from peers 6fd5746b66294aba81fddccef17a0803 (127.4.202.189:37733), 2379fdf9c84d4730b6513cb71d43c9af (127.4.202.188:34563)
I20260926 06:24:39.932266 27168 leader_election.cc:302] T 00000000000000000000000000000000 P 79be6c789b4b434daa0addb84454e41a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2379fdf9c84d4730b6513cb71d43c9af, 79be6c789b4b434daa0addb84454e41a; no voters:
I20260926 06:24:39.932376 27388 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79be6c789b4b434daa0addb84454e41a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:24:39.932519 27388 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79be6c789b4b434daa0addb84454e41a [term 1 LEADER]: Becoming Leader. State: Replica: 79be6c789b4b434daa0addb84454e41a, State: Running, Role: LEADER
I20260926 06:24:39.932608 27388 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79be6c789b4b434daa0addb84454e41a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79be6c789b4b434daa0addb84454e41a" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45711 } } peers { permanent_uuid: "6fd5746b66294aba81fddccef17a0803" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 37733 } } peers { permanent_uuid: "2379fdf9c84d4730b6513cb71d43c9af" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 34563 } }
I20260926 06:24:39.932986 27388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79be6c789b4b434daa0addb84454e41a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79be6c789b4b434daa0addb84454e41a. Latest consensus state: current_term: 1 leader_uuid: "79be6c789b4b434daa0addb84454e41a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79be6c789b4b434daa0addb84454e41a" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45711 } } peers { permanent_uuid: "6fd5746b66294aba81fddccef17a0803" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 37733 } } peers { permanent_uuid: "2379fdf9c84d4730b6513cb71d43c9af" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 34563 } } }
I20260926 06:24:39.933065 27388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79be6c789b4b434daa0addb84454e41a [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:39.933926 27406 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:24:39.934389 27406 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:24:39.932312 27150 master_runner.cc:186] Error getting master registration for 127.4.202.189:37733: OK, instance_id { permanent_uuid: "6fd5746b66294aba81fddccef17a0803" instance_seqno: 1790403879709278 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 06:24:39.934893 27150 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 06:24:39.944380 27293 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.202.188" port: 34563 } attempt: 3
I20260926 06:24:39.945850 27342 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2379fdf9c84d4730b6513cb71d43c9af [term 1 FOLLOWER]: Refusing update from remote peer 79be6c789b4b434daa0addb84454e41a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:24:39.946341 27388 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 79be6c789b4b434daa0addb84454e41a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2379fdf9c84d4730b6513cb71d43c9af" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 34563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:24:39.947618 27374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2379fdf9c84d4730b6513cb71d43c9af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79be6c789b4b434daa0addb84454e41a. Latest consensus state: current_term: 1 leader_uuid: "79be6c789b4b434daa0addb84454e41a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79be6c789b4b434daa0addb84454e41a" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45711 } } peers { permanent_uuid: "6fd5746b66294aba81fddccef17a0803" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 37733 } } peers { permanent_uuid: "2379fdf9c84d4730b6513cb71d43c9af" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 34563 } } }
I20260926 06:24:39.947710 27374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2379fdf9c84d4730b6513cb71d43c9af [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:24:39.950429 27222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.935170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42712 (local address 127.4.202.190:45711)
0926 06:24:39.935206 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:39.935207 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:39.935594 (+ 387us) server_negotiation.cc:407] Connection header received
0926 06:24:39.935802 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:39.935803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:39.935815 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:39.935895 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:39.937344 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.938063 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.940077 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.940378 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.940422 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:39.944711 (+ 4289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:39.944713 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:39.944713 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:39.944720 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:39.948852 (+ 4132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:39.949290 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:39.949292 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:39.949292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:39.949379 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:39.949607 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:39.949608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:39.949609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:39.949745 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:39.949833 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:39.950252 (+ 419us) server_negotiation.cc:299] Negotiation successful
0926 06:24:39.950339 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Sep 26 06:24:39 dist-test-slave-pd0t krb5kdc[27137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403879, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.189@KRBTEST.COM for kudu/127.4.202.188@KRBTEST.COM
I20260926 06:24:39.951373 27297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.934974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47248 (local address 127.4.202.189:37733)
0926 06:24:39.935016 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:39.935018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:39.935878 (+ 860us) server_negotiation.cc:407] Connection header received
0926 06:24:39.935991 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:39.935992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:39.936005 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:39.936038 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:39.938571 (+ 2533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.939293 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.940796 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.941090 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.941130 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:39.943239 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:39.943240 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:39.943240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:39.943247 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:39.945152 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:39.947899 (+ 2747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:39.947900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:39.947900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:39.947958 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:39.950744 (+ 2786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:39.950745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:39.950746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:39.950892 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:39.950965 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:39.951185 (+ 220us) server_negotiation.cc:299] Negotiation successful
0926 06:24:39.951266 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 06:24:39.952270 27372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.935531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51930 (local address 127.4.202.188:34563)
0926 06:24:39.935566 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:39.935568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:39.936782 (+ 1214us) server_negotiation.cc:407] Connection header received
0926 06:24:39.936875 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:39.936876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:39.936888 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:39.936915 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:39.937554 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.938211 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.940500 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.940801 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.940842 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:39.942801 (+ 1959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:39.942802 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:39.942802 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:39.942814 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:39.944616 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:39.951126 (+ 6510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:39.951127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:39.951127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:39.951198 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:39.951527 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:39.951528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:39.951529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:39.951655 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:39.951728 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:39.952070 (+ 342us) server_negotiation.cc:299] Negotiation successful
0926 06:24:39.952184 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 06:24:39.955492 27295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.945142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:34563 (local address 127.0.0.1:51936)
0926 06:24:39.945418 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:39.945428 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:39.945478 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:39.945674 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:39.945675 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:39.945856 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:39.945894 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:39.945895 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.947351 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.947352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:39.948192 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:39.948195 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.948900 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.948901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:39.948971 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:39.949440 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:24:39.949457 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:39.951145 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:39.954240 (+ 3095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:39.954241 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:39.954242 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:39.954450 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:39.954771 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:39.954772 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:39.954773 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:39.954840 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:39.955215 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:39.955216 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:39.955256 (+ 40us) client_negotiation.cc:769] Sending connection context
0926 06:24:39.955311 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 06:24:39.955399 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260}
I20260926 06:24:39.955971 27412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:39.945196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51936 (local address 127.4.202.188:34563)
0926 06:24:39.945373 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:39.945375 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:39.945466 (+ 91us) server_negotiation.cc:407] Connection header received
0926 06:24:39.945526 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:39.945526 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:39.945535 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:39.945979 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:39.946034 (+ 55us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.947221 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.948339 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:39.948796 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:39.951236 (+ 2440us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:39.951344 (+ 108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:39.951345 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:39.951345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:39.951350 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:39.954119 (+ 2769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:39.954544 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:39.954545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:39.954545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:39.954679 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:39.954920 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:39.954921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:39.954921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:39.955109 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:39.955609 (+ 500us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:39.955712 (+ 103us) server_negotiation.cc:299] Negotiation successful
0926 06:24:39.955811 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":78,"threads_started":1}
I20260926 06:24:39.957432 27293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fd5746b66294aba81fddccef17a0803: Bootstrap starting.
I20260926 06:24:39.958102 27293 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6fd5746b66294aba81fddccef17a0803: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:39.958395 27293 log.cc:824] T 00000000000000000000000000000000 P 6fd5746b66294aba81fddccef17a0803: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:39.958949 27388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79be6c789b4b434daa0addb84454e41a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "79be6c789b4b434daa0addb84454e41a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79be6c789b4b434daa0addb84454e41a" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45711 } } peers { permanent_uuid: "6fd5746b66294aba81fddccef17a0803" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 37733 } } peers { permanent_uuid: "2379fdf9c84d4730b6513cb71d43c9af" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 34563 } } }
I20260926 06:24:39.959026 27388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79be6c789b4b434daa0addb84454e41a [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:39.959151 27408 mvcc.cc:205] Tried to move back new op lower bound from 7333494292253888512 to 7333494292204531712. Current Snapshot: MvccSnapshot[applied={T|T < 7333494292253888512}]
I20260926 06:24:39.959254 27293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fd5746b66294aba81fddccef17a0803: No bootstrap required, opened a new log
I20260926 06:24:39.959568 27374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2379fdf9c84d4730b6513cb71d43c9af [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "79be6c789b4b434daa0addb84454e41a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79be6c789b4b434daa0addb84454e41a" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45711 } } peers { permanent_uuid: "6fd5746b66294aba81fddccef17a0803" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 37733 } } peers { permanent_uuid: "2379fdf9c84d4730b6513cb71d43c9af" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 34563 } } }
I20260926 06:24:39.959651 27374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2379fdf9c84d4730b6513cb71d43c9af [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:24:39.960427 27413 mvcc.cc:205] Tried to move back new op lower bound from 7333494292253888512 to 7333494292204531712. Current Snapshot: MvccSnapshot[applied={T|T < 7333494292253888512}]
I20260926 06:24:39.961169 27293 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6fd5746b66294aba81fddccef17a0803 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79be6c789b4b434daa0addb84454e41a" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45711 } } peers { permanent_uuid: "6fd5746b66294aba81fddccef17a0803" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 37733 } } peers { permanent_uuid: "2379fdf9c84d4730b6513cb71d43c9af" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 34563 } }
I20260926 06:24:39.961308 27293 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6fd5746b66294aba81fddccef17a0803 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fd5746b66294aba81fddccef17a0803, State: Initialized, Role: FOLLOWER
I20260926 06:24:39.961386 27293 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6fd5746b66294aba81fddccef17a0803 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79be6c789b4b434daa0addb84454e41a" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45711 } } peers { permanent_uuid: "6fd5746b66294aba81fddccef17a0803" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 37733 } } peers { permanent_uuid: "2379fdf9c84d4730b6513cb71d43c9af" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 34563 } }
I20260926 06:24:39.961705 27266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "79be6c789b4b434daa0addb84454e41a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6fd5746b66294aba81fddccef17a0803"
I20260926 06:24:39.961755 27267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6fd5746b66294aba81fddccef17a0803 [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:39.962517 27267 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6fd5746b66294aba81fddccef17a0803 [term 1 FOLLOWER]: Refusing update from remote peer 79be6c789b4b434daa0addb84454e41a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:24:39.962915 27400 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 79be6c789b4b434daa0addb84454e41a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fd5746b66294aba81fddccef17a0803" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 37733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:24:39.961674 27293 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6fd5746b66294aba81fddccef17a0803 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:24:39.963587 27266 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6fd5746b66294aba81fddccef17a0803 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79be6c789b4b434daa0addb84454e41a in term 1.
I20260926 06:24:39.965837 27428 mvcc.cc:205] Tried to move back new op lower bound from 7333494292253888512 to 7333494292204531712. Current Snapshot: MvccSnapshot[applied={T|T < 7333494292253888512}]
I20260926 06:24:39.965977 27423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fd5746b66294aba81fddccef17a0803 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "79be6c789b4b434daa0addb84454e41a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79be6c789b4b434daa0addb84454e41a" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45711 } } peers { permanent_uuid: "6fd5746b66294aba81fddccef17a0803" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 37733 } } peers { permanent_uuid: "2379fdf9c84d4730b6513cb71d43c9af" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 34563 } } }
I20260926 06:24:39.966110 27423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fd5746b66294aba81fddccef17a0803 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:24:39.967036 27406 catalog_manager.cc:1347] Generated new cluster ID: 1d5cb060711b473c8f54cd03cf8221c9
I20260926 06:24:39.967094 27406 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
W20260926 06:24:39.967069 27433 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6fd5746b66294aba81fddccef17a0803: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:24:39.967242 27433 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:24:39.961661 27418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fd5746b66294aba81fddccef17a0803 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79be6c789b4b434daa0addb84454e41a" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45711 } } peers { permanent_uuid: "6fd5746b66294aba81fddccef17a0803" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 37733 } } peers { permanent_uuid: "2379fdf9c84d4730b6513cb71d43c9af" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 34563 } } }
I20260926 06:24:39.964859 27432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fd5746b66294aba81fddccef17a0803 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79be6c789b4b434daa0addb84454e41a. Latest consensus state: current_term: 1 leader_uuid: "79be6c789b4b434daa0addb84454e41a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79be6c789b4b434daa0addb84454e41a" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45711 } } peers { permanent_uuid: "6fd5746b66294aba81fddccef17a0803" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 37733 } } peers { permanent_uuid: "2379fdf9c84d4730b6513cb71d43c9af" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 34563 } } }
I20260926 06:24:39.967376 27432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fd5746b66294aba81fddccef17a0803 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:24:39.967502 27418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fd5746b66294aba81fddccef17a0803 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:24:39.975035 27400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79be6c789b4b434daa0addb84454e41a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "79be6c789b4b434daa0addb84454e41a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79be6c789b4b434daa0addb84454e41a" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 45711 } } peers { permanent_uuid: "6fd5746b66294aba81fddccef17a0803" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 37733 } } peers { permanent_uuid: "2379fdf9c84d4730b6513cb71d43c9af" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 34563 } } }
I20260926 06:24:39.975126 27400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79be6c789b4b434daa0addb84454e41a [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:39.977259 27406 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:24:39.977967 27406 catalog_manager.cc:1504] Loading token signing keys...
I20260926 06:24:39.995524 27406 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79be6c789b4b434daa0addb84454e41a: Generated new TSK 0
I20260926 06:24:39.995678 27406 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 06:24:40.024757 27222 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:24:40.023368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42726 (local address 127.4.202.190:45711)
0926 06:24:40.023408 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:40.023410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:40.024175 (+ 765us) server_negotiation.cc:407] Connection header received
0926 06:24:40.024269 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:40.024270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:40.024281 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:40.024322 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:40.024696 (+ 374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42726: BlockingRecv error: recv got EOF from 127.0.0.1:42726 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260926 06:24:40.025472 27412 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:24:40.023658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51940 (local address 127.4.202.188:34563)
0926 06:24:40.023694 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:40.023695 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:40.024969 (+ 1274us) server_negotiation.cc:407] Connection header received
0926 06:24:40.025057 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:40.025058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:40.025070 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:40.025098 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:40.025413 (+ 315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51940: BlockingRecv error: recv got EOF from 127.0.0.1:51940 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260926 06:24:40.026458 27297 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:24:40.023945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47264 (local address 127.4.202.189:37733)
0926 06:24:40.023979 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:40.023981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:40.025916 (+ 1935us) server_negotiation.cc:407] Connection header received
0926 06:24:40.026057 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:40.026058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:40.026068 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:40.026096 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:40.026402 (+ 306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47264: BlockingRecv error: recv got EOF from 127.0.0.1:47264 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260926 06:24:40.028658 27297 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:24:40.027949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47278 (local address 127.4.202.189:37733)
0926 06:24:40.027980 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:40.027981 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:40.028149 (+ 168us) server_negotiation.cc:407] Connection header received
0926 06:24:40.028240 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:40.028241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:40.028256 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:40.028283 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:40.028618 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47278: BlockingRecv error: recv got EOF from 127.0.0.1:47278 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260926 06:24:40.031596 27297 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:24:40.030856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47288 (local address 127.4.202.189:37733)
0926 06:24:40.030906 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:40.030908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:40.031121 (+ 213us) server_negotiation.cc:407] Connection header received
0926 06:24:40.031203 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:40.031204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:40.031213 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:40.031239 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:40.031558 (+ 319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47288: BlockingRecv error: recv got EOF from 127.0.0.1:47288 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260926 06:24:40.035462 27297 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:24:40.034763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47304 (local address 127.4.202.189:37733)
0926 06:24:40.034795 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:40.034797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:40.035010 (+ 213us) server_negotiation.cc:407] Connection header received
0926 06:24:40.035094 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:40.035095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:40.035104 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:40.035130 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:40.035425 (+ 295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47304: BlockingRecv error: recv got EOF from 127.0.0.1:47304 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260926 06:24:40.041435 27297 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:24:40.040555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47308 (local address 127.4.202.189:37733)
0926 06:24:40.040586 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:40.040588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:40.040887 (+ 299us) server_negotiation.cc:407] Connection header received
0926 06:24:40.040981 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:40.040981 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:40.040991 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:40.041020 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:40.041396 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47308: BlockingRecv error: recv got EOF from 127.0.0.1:47308 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260926 06:24:40.051362 27297 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:24:40.050511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47324 (local address 127.4.202.189:37733)
0926 06:24:40.050544 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:40.050546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:40.050848 (+ 302us) server_negotiation.cc:407] Connection header received
0926 06:24:40.050943 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:40.050944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:40.050954 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:40.050983 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:40.051317 (+ 334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47324: BlockingRecv error: recv got EOF from 127.0.0.1:47324 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260926 06:24:40.069394 27297 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:24:40.068539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47336 (local address 127.4.202.189:37733)
0926 06:24:40.068574 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:40.068576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:40.068829 (+ 253us) server_negotiation.cc:407] Connection header received
0926 06:24:40.068934 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:40.068935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:40.068947 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:40.068977 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:40.069340 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47336: BlockingRecv error: recv got EOF from 127.0.0.1:47336 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260926 06:24:40.103555 27297 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:24:40.102729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47346 (local address 127.4.202.189:37733)
0926 06:24:40.102771 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:40.102773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:40.102980 (+ 207us) server_negotiation.cc:407] Connection header received
0926 06:24:40.103021 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:40.103023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:40.103040 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:40.103125 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:40.103497 (+ 372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47346: BlockingRecv error: recv got EOF from 127.0.0.1:47346 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260926 06:24:40.169512 27297 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:24:40.168711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47350 (local address 127.4.202.189:37733)
0926 06:24:40.168848 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:40.168850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:40.168945 (+ 95us) server_negotiation.cc:407] Connection header received
0926 06:24:40.168978 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:40.168979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:40.168990 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:40.169053 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:40.169425 (+ 372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47350: BlockingRecv error: recv got EOF from 127.0.0.1:47350 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
W20260926 06:24:40.299355 27297 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:24:40.298681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47364 (local address 127.4.202.189:37733)
0926 06:24:40.298752 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:40.298788 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:40.298831 (+ 43us) server_negotiation.cc:407] Connection header received
0926 06:24:40.298891 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:40.298892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:40.298903 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:40.298970 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:40.299306 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47364: BlockingRecv error: recv got EOF from 127.0.0.1:47364 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260926 06:24:40.557372 27297 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:24:40.556644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47380 (local address 127.4.202.189:37733)
0926 06:24:40.556693 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:40.556695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:40.556810 (+ 115us) server_negotiation.cc:407] Connection header received
0926 06:24:40.556892 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:40.556893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:40.556902 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:40.556978 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:40.557310 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47380: BlockingRecv error: recv got EOF from 127.0.0.1:47380 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:24:40.888599 27385 catalog_manager.cc:1259] Loaded cluster ID: 1d5cb060711b473c8f54cd03cf8221c9
I20260926 06:24:40.888665 27385 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2379fdf9c84d4730b6513cb71d43c9af: loading cluster ID for follower catalog manager: success
I20260926 06:24:40.889504 27385 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2379fdf9c84d4730b6513cb71d43c9af: acquiring CA information for follower catalog manager: success
I20260926 06:24:40.889816 27385 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2379fdf9c84d4730b6513cb71d43c9af: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 06:24:40.967691 27433 catalog_manager.cc:1259] Loaded cluster ID: 1d5cb060711b473c8f54cd03cf8221c9
I20260926 06:24:40.967770 27433 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6fd5746b66294aba81fddccef17a0803: loading cluster ID for follower catalog manager: success
I20260926 06:24:40.968629 27433 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6fd5746b66294aba81fddccef17a0803: acquiring CA information for follower catalog manager: success
I20260926 06:24:40.968943 27433 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6fd5746b66294aba81fddccef17a0803: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 06:24:41.072956 27501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:41.070644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47384 (local address 127.4.202.189:37733)
0926 06:24:41.070763 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:41.070765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:41.070801 (+ 36us) server_negotiation.cc:407] Connection header received
0926 06:24:41.070834 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:41.070835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:41.070881 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:41.070951 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:24:41.071088 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:41.071569 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:41.072292 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:41.072490 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:41.072540 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:41.072675 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:24:41.072777 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:24:41.072841 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:41.072874 (+ 33us) server_negotiation.cc:299] Negotiation successful
0926 06:24:41.072892 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":67,"threads_started":1}
I20260926 06:24:41.076948 27507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:41.074716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51954 (local address 127.4.202.188:34563)
0926 06:24:41.074803 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:41.074805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:41.074816 (+ 11us) server_negotiation.cc:407] Connection header received
0926 06:24:41.074875 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:41.074876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:41.074907 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:41.074970 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:24:41.075045 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:41.075500 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:41.076205 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:41.076400 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:41.076451 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:41.076578 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:24:41.076663 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:24:41.076717 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:41.076800 (+ 83us) server_negotiation.cc:299] Negotiation successful
0926 06:24:41.076819 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68,"thread_start_us":45,"threads_started":1}
I20260926 06:24:41.081332 27501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:41.078507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47398 (local address 127.4.202.189:37733)
0926 06:24:41.078552 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:41.078554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:41.078692 (+ 138us) server_negotiation.cc:407] Connection header received
0926 06:24:41.078828 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:41.078829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:41.078839 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:41.078890 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:24:41.079089 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:41.079579 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:41.080572 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:41.080821 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:41.080903 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:41.081068 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:24:41.081099 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:24:41.081124 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:41.081217 (+ 93us) server_negotiation.cc:299] Negotiation successful
0926 06:24:41.081241 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 06:24:41.081787 27513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:41.078455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42742 (local address 127.4.202.190:45711)
0926 06:24:41.078564 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:41.078566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:41.078737 (+ 171us) server_negotiation.cc:407] Connection header received
0926 06:24:41.078786 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:41.078787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:41.078874 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:41.078947 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:24:41.079125 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:41.079887 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:41.080664 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:41.080974 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:41.081052 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:41.081216 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:24:41.081410 (+ 194us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:24:41.081477 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:41.081675 (+ 198us) server_negotiation.cc:299] Negotiation successful
0926 06:24:41.081705 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":56,"threads_started":1}
I20260926 06:24:41.081842 27507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:41.078687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51956 (local address 127.4.202.188:34563)
0926 06:24:41.078728 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:41.078730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:41.078858 (+ 128us) server_negotiation.cc:407] Connection header received
0926 06:24:41.078919 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:41.078920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:41.078931 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:41.078974 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:24:41.079103 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:41.079862 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:41.080891 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:41.081181 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:41.081239 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:41.081600 (+ 361us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:24:41.081636 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:24:41.081686 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:41.081733 (+ 47us) server_negotiation.cc:299] Negotiation successful
0926 06:24:41.081755 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 06:24:41.083703 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 27150
I20260926 06:24:41.089115 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 27225
I20260926 06:24:41.093796 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 27300
2026-09-26T06:24:41Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2809 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 06:24:41 dist-test-slave-pd0t krb5kdc[27518](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 06:24:41 dist-test-slave-pd0t krb5kdc[27518](info): set up 2 sockets
Sep 26 06:24:41 dist-test-slave-pd0t krb5kdc[27518](info): commencing operation
krb5kdc: starting...
W20260926 06:24:42.118803 4906 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/krb5kdc/test-user.keytab.
Sep 26 06:24:42 dist-test-slave-pd0t krb5kdc[27518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T06:24:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T06:24:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.202.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
Entry for principal kudu/127.4.202.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab.
I20260926 06:24:42.234290 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:38437
--webserver_interface=127.4.202.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:35979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:38437,127.4.202.189:32903,127.4.202.188:37851
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 06:24:42.312565 27531 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 06:24:42.312783 27531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:42.312799 27531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:42.312851 27531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:42.314047 27531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:24:42.314088 27531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:42.314105 27531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:42.314118 27531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:24:42.314152 27531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:24:42.315632 27531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:35979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:38437,127.4.202.189:32903,127.4.202.188:37851
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.190.keytab
--principal=kudu/127.4.202.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.190:38437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.202.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:42.315884 27531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:42.316115 27531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:42.320879 27536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:42.320924 27537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:42.320922 27539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:42.320973 27531 server_base.cc:1034] running on GCE node
I20260926 06:24:42.321249 27531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:42.321537 27531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:42.322675 27531 hybrid_clock.cc:648] HybridClock initialized: now 1790403882322648 us; error 37 us; skew 500 ppm
Sep 26 06:24:42 dist-test-slave-pd0t krb5kdc[27518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403882, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:42.325169 27531 init.cc:377] Logged in from keytab as kudu/127.4.202.190@KRBTEST.COM (short username kudu)
I20260926 06:24:42.325569 27531 webserver.cc:460] Webserver started at http://127.4.202.190:36329/ using document root <none> and password file <none>
I20260926 06:24:42.325814 27531 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:42.325889 27531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:42.326031 27531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:42.327102 27531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-0/data/instance:
uuid: "2871dc28296a4a0684c614f519e74abe"
format_stamp: "Formatted at 2026-09-26 06:24:42 on dist-test-slave-pd0t"
server_key: "f3b7de158510e638e4aabb7d0fb43cc5"
server_key_iv: "08f2c3ada1aaf55e610239329af6df25"
server_key_version: "encryptionkey@0"
I20260926 06:24:42.327437 27531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-0/wal/instance:
uuid: "2871dc28296a4a0684c614f519e74abe"
format_stamp: "Formatted at 2026-09-26 06:24:42 on dist-test-slave-pd0t"
server_key: "f3b7de158510e638e4aabb7d0fb43cc5"
server_key_iv: "08f2c3ada1aaf55e610239329af6df25"
server_key_version: "encryptionkey@0"
I20260926 06:24:42.328704 27531 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 06:24:42.329372 27546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:42.329532 27531 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:24:42.329633 27531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-0/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-0/wal
uuid: "2871dc28296a4a0684c614f519e74abe"
format_stamp: "Formatted at 2026-09-26 06:24:42 on dist-test-slave-pd0t"
server_key: "f3b7de158510e638e4aabb7d0fb43cc5"
server_key_iv: "08f2c3ada1aaf55e610239329af6df25"
server_key_version: "encryptionkey@0"
I20260926 06:24:42.329703 27531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:42.345232 27531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:42.347251 27531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:42.347409 27531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:42.351166 27531 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.190:38437
I20260926 06:24:42.351198 27598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.190:38437 every 8 connection(s)
I20260926 06:24:42.351536 27531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-0/data/info.pb
I20260926 06:24:42.352514 27599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:42.353965 27599 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.202.190" port: 38437 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:24:42.358341 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 27531
I20260926 06:24:42.358449 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-0/wal/instance
I20260926 06:24:42.358669 4906 external_mini_cluster.cc:1428] Setting key d99df43faf3acc12ce809157259e16ef
Sep 26 06:24:42 dist-test-slave-pd0t krb5kdc[27518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403882, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:24:42.363202 27602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.354653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48540 (local address 127.4.202.190:38437)
0926 06:24:42.354788 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:42.354795 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:42.354950 (+ 155us) server_negotiation.cc:407] Connection header received
0926 06:24:42.355402 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:42.355403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:42.355419 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:42.355493 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:42.355977 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.356756 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.357555 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.357842 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.357927 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:42.360119 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:42.360125 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:42.360126 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:42.360132 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:42.361602 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:42.362058 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:42.362060 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:42.362061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:42.362132 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:42.362386 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:42.362387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:42.362387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:42.362569 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:42.362719 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:42.362971 (+ 252us) server_negotiation.cc:299] Negotiation successful
0926 06:24:42.363063 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":63,"threads_started":1}
I20260926 06:24:42.363337 27601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.354620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:38437 (local address 127.0.0.1:48540)
0926 06:24:42.354839 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:42.354864 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:42.355279 (+ 415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:42.355522 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:42.355524 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:42.355747 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:42.355894 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:42.355898 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.356833 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.356834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:42.357487 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:42.357489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.357935 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.357936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:42.357987 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:42.358515 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:24:42.358525 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:42.360037 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:42.361688 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:42.361695 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:42.361697 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:42.361976 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:42.362203 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:42.362204 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:42.362204 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:42.362314 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:42.362732 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:42.362740 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:42.362822 (+ 82us) client_negotiation.cc:769] Sending connection context
0926 06:24:42.362937 (+ 115us) client_negotiation.cc:240] Negotiation successful
0926 06:24:42.363166 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":67,"thread_start_us":85,"threads_started":1}
I20260926 06:24:42.364063 27599 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.202.189" port: 32903 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:24:42.364411 27601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.364277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:32903 (local address 127.0.0.1:46156)
0926 06:24:42.364328 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:42.364381 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.189:32903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260926 06:24:42.364553 27549 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.202.189:32903: connect: Connection refused (error 111)
I20260926 06:24:42.364964 27601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.364854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:32903 (local address 127.0.0.1:46158)
0926 06:24:42.364904 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:42.364936 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.189:32903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260926 06:24:42.365073 27599 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.202.189:32903: Network error: Client connection negotiation failed: client connection to 127.4.202.189:32903: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.4.202.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.189.keytab.
Entry for principal kudu/127.4.202.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.189.keytab.
I20260926 06:24:42.382571 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.189:32903
--webserver_interface=127.4.202.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.189.keytab
--principal=kudu/127.4.202.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:35979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:38437,127.4.202.189:32903,127.4.202.188:37851
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:24:42.414294 27599 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.202.189" port: 32903 } attempt: 1
I20260926 06:24:42.414845 27601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.414694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:32903 (local address 127.0.0.1:46168)
0926 06:24:42.414756 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:42.414803 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.189:32903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260926 06:24:42.415236 27601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.415145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:32903 (local address 127.0.0.1:46182)
0926 06:24:42.415190 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:42.415217 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.189:32903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260926 06:24:42.415350 27599 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.202.189:32903: Network error: Client connection negotiation failed: client connection to 127.4.202.189:32903: connect: Connection refused (error 111)
W20260926 06:24:42.462148 27606 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 06:24:42.462404 27606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:42.462421 27606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:42.462441 27606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:42.463634 27606 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:24:42.463676 27606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:42.463691 27606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:42.463703 27606 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:24:42.463762 27606 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:24:42.465222 27606 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:35979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:38437,127.4.202.189:32903,127.4.202.188:37851
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.189.keytab
--principal=kudu/127.4.202.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.189:32903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.202.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:42.465479 27606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:42.465708 27606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:42.470345 27611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:42.470331 27612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:42.470319 27614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:42.470599 27606 server_base.cc:1034] running on GCE node
I20260926 06:24:42.470790 27606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:42.471107 27606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:42.472270 27606 hybrid_clock.cc:648] HybridClock initialized: now 1790403882472260 us; error 26 us; skew 500 ppm
Sep 26 06:24:42 dist-test-slave-pd0t krb5kdc[27518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403882, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:42.474797 27606 init.cc:377] Logged in from keytab as kudu/127.4.202.189@KRBTEST.COM (short username kudu)
I20260926 06:24:42.475222 27606 webserver.cc:460] Webserver started at http://127.4.202.189:36813/ using document root <none> and password file <none>
I20260926 06:24:42.475440 27606 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:42.475483 27606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:42.475620 27606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:42.476650 27606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-1/data/instance:
uuid: "939f962d1c244a8b92f48154f30e5562"
format_stamp: "Formatted at 2026-09-26 06:24:42 on dist-test-slave-pd0t"
server_key: "24de1dad35c4e5bfc25f27794fb68585"
server_key_iv: "9f1968baa7e2e36cf1f819e6a7a4b619"
server_key_version: "encryptionkey@0"
I20260926 06:24:42.477001 27606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-1/wal/instance:
uuid: "939f962d1c244a8b92f48154f30e5562"
format_stamp: "Formatted at 2026-09-26 06:24:42 on dist-test-slave-pd0t"
server_key: "24de1dad35c4e5bfc25f27794fb68585"
server_key_iv: "9f1968baa7e2e36cf1f819e6a7a4b619"
server_key_version: "encryptionkey@0"
I20260926 06:24:42.478274 27606 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260926 06:24:42.479216 27621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:42.480068 27606 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 06:24:42.480165 27606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-1/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-1/wal
uuid: "939f962d1c244a8b92f48154f30e5562"
format_stamp: "Formatted at 2026-09-26 06:24:42 on dist-test-slave-pd0t"
server_key: "24de1dad35c4e5bfc25f27794fb68585"
server_key_iv: "9f1968baa7e2e36cf1f819e6a7a4b619"
server_key_version: "encryptionkey@0"
I20260926 06:24:42.480258 27606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:42.483495 27599 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.202.189" port: 32903 } attempt: 2
I20260926 06:24:42.483884 27601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.483753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:32903 (local address 127.0.0.1:46186)
0926 06:24:42.483808 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:42.483842 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.189:32903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 06:24:42.484225 27601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.484139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:32903 (local address 127.0.0.1:46190)
0926 06:24:42.484174 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:42.484197 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.189:32903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260926 06:24:42.484323 27599 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.202.189:32903: Network error: Client connection negotiation failed: client connection to 127.4.202.189:32903: connect: Connection refused (error 111)
I20260926 06:24:42.507979 27606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:42.510056 27606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:42.510202 27606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:42.514256 27606 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.189:32903
I20260926 06:24:42.514298 27673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.189:32903 every 8 connection(s)
I20260926 06:24:42.514678 27606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-1/data/info.pb
I20260926 06:24:42.515688 27674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:42.516908 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 27606
I20260926 06:24:42.517014 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-1/wal/instance
I20260926 06:24:42.517213 4906 external_mini_cluster.cc:1428] Setting key 0ef437871feecf95e8750d53659cafaf
I20260926 06:24:42.517380 27674 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.202.190" port: 38437 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 06:24:42 dist-test-slave-pd0t krb5kdc[27518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403882, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.189@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.202.188@KRBTEST.COM; defaulting to no policy
I20260926 06:24:42.526675 27602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.518106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48552 (local address 127.4.202.190:38437)
0926 06:24:42.518175 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:42.518177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:42.518447 (+ 270us) server_negotiation.cc:407] Connection header received
0926 06:24:42.519023 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:42.519024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:42.519040 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:42.519114 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:42.519705 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.520225 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.521219 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.521440 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.521528 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:42.523650 (+ 2122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:42.523652 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:42.523652 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:42.523659 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:42.525119 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:42.525615 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:42.525617 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:42.525617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:42.525681 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:42.525975 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:42.525976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:42.525976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:42.526089 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:42.526165 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:42.526474 (+ 309us) server_negotiation.cc:299] Negotiation successful
0926 06:24:42.526558 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 06:24:42.526850 27676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.518087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:38437 (local address 127.0.0.1:48552)
0926 06:24:42.518282 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:42.518344 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:42.518904 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:42.519181 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:42.519184 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:42.519428 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:42.519622 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:42.519627 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.520333 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.520334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:42.521126 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:42.521129 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.521554 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.521556 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:42.521679 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:42.522173 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:24:42.522181 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:42.523536 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:42.525222 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:42.525231 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:42.525232 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:42.525532 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:42.525768 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:42.525769 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:42.525769 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:42.525898 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:42.526157 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:42.526164 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:42.526340 (+ 176us) client_negotiation.cc:769] Sending connection context
0926 06:24:42.526453 (+ 113us) client_negotiation.cc:240] Negotiation successful
0926 06:24:42.526625 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":99,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.202.188@KRBTEST.COM" created.
I20260926 06:24:42.528267 27674 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.202.189" port: 32903 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 06:24:42 dist-test-slave-pd0t krb5kdc[27518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403882, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.189@KRBTEST.COM for kudu/127.4.202.189@KRBTEST.COM
I20260926 06:24:42.536427 27676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.528540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:32903 (local address 127.0.0.1:46204)
0926 06:24:42.528599 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:42.528612 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:42.528661 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:42.528867 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:42.528868 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:42.529028 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:42.529088 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:42.529089 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.529949 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.529951 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:42.530852 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:42.530855 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.531297 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.531298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:42.531383 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:42.531721 (+ 338us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:24:42.531727 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:42.533336 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:42.535201 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:42.535202 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:42.535202 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:42.535401 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:42.535591 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:42.535592 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:42.535592 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:42.535641 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:42.536183 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:42.536184 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:42.536220 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 06:24:42.536287 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 06:24:42.536366 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260926 06:24:42.536580 27679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.528577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46204 (local address 127.4.202.189:32903)
0926 06:24:42.528711 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:42.528716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:42.528733 (+ 17us) server_negotiation.cc:407] Connection header received
0926 06:24:42.528761 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:42.528763 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:42.528773 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:42.528840 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:42.529160 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.529857 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.530936 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.531218 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.531274 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:42.533513 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:42.533520 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:42.533520 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:42.533526 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:42.535127 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:42.535475 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:42.535477 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:42.535477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:42.535530 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:42.535721 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:42.535722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:42.535722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:42.536093 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:42.536172 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:42.536332 (+ 160us) server_negotiation.cc:299] Negotiation successful
0926 06:24:42.536447 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":108,"thread_start_us":83,"threads_started":1}
I20260926 06:24:42.537112 27674 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.202.188" port: 37851 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:24:42.537495 27676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.537372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:37851 (local address 127.0.0.1:41130)
0926 06:24:42.537421 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:42.537469 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.188:37851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260926 06:24:42.537652 27624 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.202.188:37851: connect: Connection refused (error 111)
I20260926 06:24:42.538120 27676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.538029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:37851 (local address 127.0.0.1:41142)
0926 06:24:42.538077 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:42.538103 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.188:37851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260926 06:24:42.538215 27674 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.202.188:37851: Network error: Client connection negotiation failed: client connection to 127.4.202.188:37851: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.202.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.188.keytab.
Entry for principal kudu/127.4.202.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.188.keytab.
I20260926 06:24:42.541796 4906 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm7eEJS/build/release/bin/kudu
/tmp/dist-test-taskm7eEJS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.188:37851
--webserver_interface=127.4.202.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.188.keytab
--principal=kudu/127.4.202.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.202.148:35979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.202.190:38437,127.4.202.189:32903,127.4.202.188:37851
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 06:24:42.575459 27599 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.202.189" port: 32903 } attempt: 3
Sep 26 06:24:42 dist-test-slave-pd0t krb5kdc[27518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403882, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for kudu/127.4.202.189@KRBTEST.COM
I20260926 06:24:42.582337 27601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.575768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:32903 (local address 127.0.0.1:46212)
0926 06:24:42.575829 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:42.575842 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:42.575894 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:42.576048 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:42.576049 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:42.576200 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:42.576260 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:42.576262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.576952 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.576953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:42.577487 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:42.577489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.577909 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.577910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:42.577996 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:42.578467 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:24:42.578472 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:42.579680 (+ 1208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:42.581409 (+ 1729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:42.581409 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:42.581410 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:42.581568 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:42.581764 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:42.581765 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:42.581765 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:42.581815 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:42.582102 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:42.582103 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:42.582141 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 06:24:42.582196 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 06:24:42.582265 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260926 06:24:42.582403 27679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.575851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46212 (local address 127.4.202.189:32903)
0926 06:24:42.575901 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:42.575903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:42.575915 (+ 12us) server_negotiation.cc:407] Connection header received
0926 06:24:42.575945 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:42.575946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:42.575959 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:42.576026 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:42.576378 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.576865 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.577587 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.577810 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.577910 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:42.579764 (+ 1854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:42.579765 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:42.579766 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:42.579775 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:42.581323 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:42.581636 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:42.581638 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:42.581638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:42.581699 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:42.581895 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:42.581896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:42.581896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:42.582031 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:42.582097 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:42.582238 (+ 141us) server_negotiation.cc:299] Negotiation successful
0926 06:24:42.582316 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 06:24:42.582703 27599 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.202.188" port: 37851 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:24:42.583030 27601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.582929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:37851 (local address 127.0.0.1:41150)
0926 06:24:42.582977 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:42.583010 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.188:37851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260926 06:24:42.583427 27601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.583313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:37851 (local address 127.0.0.1:41156)
0926 06:24:42.583364 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:42.583398 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.188:37851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260926 06:24:42.583540 27599 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.202.188:37851: Network error: Client connection negotiation failed: client connection to 127.4.202.188:37851: connect: Connection refused (error 111)
I20260926 06:24:42.587450 27674 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.202.188" port: 37851 } attempt: 1
I20260926 06:24:42.587795 27676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.587688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:37851 (local address 127.0.0.1:41168)
0926 06:24:42.587738 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:42.587769 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.188:37851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260926 06:24:42.588101 27676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.588011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:37851 (local address 127.0.0.1:41184)
0926 06:24:42.588053 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:42.588081 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.188:37851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260926 06:24:42.588194 27674 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.202.188:37851: Network error: Client connection negotiation failed: client connection to 127.4.202.188:37851: connect: Connection refused (error 111)
I20260926 06:24:42.614651 27599 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.202.188" port: 37851 } attempt: 1
I20260926 06:24:42.615118 27601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.614971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:37851 (local address 127.0.0.1:41192)
0926 06:24:42.615035 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:42.615078 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.188:37851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260926 06:24:42.615525 27601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.615425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:37851 (local address 127.0.0.1:41194)
0926 06:24:42.615468 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:42.615499 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.188:37851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260926 06:24:42.615629 27599 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.202.188:37851: Network error: Client connection negotiation failed: client connection to 127.4.202.188:37851: connect: Connection refused (error 111)
W20260926 06:24:42.626075 27681 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 06:24:42.626260 27681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 06:24:42.626277 27681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 06:24:42.626297 27681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 06:24:42.627436 27681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 06:24:42.627475 27681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 06:24:42.627488 27681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 06:24:42.627501 27681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 06:24:42.627512 27681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 06:24:42.628866 27681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.202.148:35979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.202.190:38437,127.4.202.189:32903,127.4.202.188:37851
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/krb5kdc/kudu_127.4.202.188.keytab
--principal=kudu/127.4.202.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.202.188:37851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.202.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 05:33:27 UTC on ccycloud.quasar-vdapsj.root.comops.site
build id 4673735
I20260926 06:24:42.629040 27681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 06:24:42.629194 27681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 06:24:42.633786 27687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:42.633852 27686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 06:24:42.633852 27689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 06:24:42.634073 27681 server_base.cc:1034] running on GCE node
I20260926 06:24:42.634312 27681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 06:24:42.634639 27681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 06:24:42.635785 27681 hybrid_clock.cc:648] HybridClock initialized: now 1790403882635760 us; error 42 us; skew 500 ppm
Sep 26 06:24:42 dist-test-slave-pd0t krb5kdc[27518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403882, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 06:24:42.638034 27681 init.cc:377] Logged in from keytab as kudu/127.4.202.188@KRBTEST.COM (short username kudu)
I20260926 06:24:42.638434 27681 webserver.cc:460] Webserver started at http://127.4.202.188:39045/ using document root <none> and password file <none>
I20260926 06:24:42.638731 27681 fs_manager.cc:359] Metadata directory not provided
I20260926 06:24:42.638795 27681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 06:24:42.638921 27681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 06:24:42.639891 27681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-2/data/instance:
uuid: "252498b9bfc14e8cb61b0bd1acc4e7c9"
format_stamp: "Formatted at 2026-09-26 06:24:42 on dist-test-slave-pd0t"
server_key: "17620dbc2ed22af59c9891c725333acf"
server_key_iv: "ab634a1e5b0b0d1c4b19d23305df16e6"
server_key_version: "encryptionkey@0"
I20260926 06:24:42.640241 27681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-2/wal/instance:
uuid: "252498b9bfc14e8cb61b0bd1acc4e7c9"
format_stamp: "Formatted at 2026-09-26 06:24:42 on dist-test-slave-pd0t"
server_key: "17620dbc2ed22af59c9891c725333acf"
server_key_iv: "ab634a1e5b0b0d1c4b19d23305df16e6"
server_key_version: "encryptionkey@0"
I20260926 06:24:42.641505 27681 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 06:24:42.642160 27696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 06:24:42.642321 27681 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 06:24:42.642421 27681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-2/data,/tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-2/wal
uuid: "252498b9bfc14e8cb61b0bd1acc4e7c9"
format_stamp: "Formatted at 2026-09-26 06:24:42 on dist-test-slave-pd0t"
server_key: "17620dbc2ed22af59c9891c725333acf"
server_key_iv: "ab634a1e5b0b0d1c4b19d23305df16e6"
server_key_version: "encryptionkey@0"
I20260926 06:24:42.642505 27681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 06:24:42.654358 27681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 06:24:42.656080 27681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 06:24:42.656232 27681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 06:24:42.656365 27674 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.202.188" port: 37851 } attempt: 2
I20260926 06:24:42.656802 27676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.656652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:37851 (local address 127.0.0.1:41210)
0926 06:24:42.656724 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:42.656765 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.188:37851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260926 06:24:42.657195 27676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.657081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:37851 (local address 127.0.0.1:41214)
0926 06:24:42.657135 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:42.657168 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.202.188:37851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260926 06:24:42.657317 27674 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.202.188:37851: Network error: Client connection negotiation failed: client connection to 127.4.202.188:37851: connect: Connection refused (error 111)
I20260926 06:24:42.660348 27681 rpc_server.cc:307] RPC server started. Bound to: 127.4.202.188:37851
I20260926 06:24:42.660378 27748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.202.188:37851 every 8 connection(s)
I20260926 06:24:42.660697 27681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-2/data/info.pb
I20260926 06:24:42.661595 27749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 06:24:42.663089 27749 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.202.190" port: 38437 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:24:42.666019 4906 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm7eEJS/build/release/bin/kudu as pid 27681
I20260926 06:24:42.666143 4906 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm7eEJS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790403591640726-4906-0/minicluster-data/master-2/wal/instance
I20260926 06:24:42.666348 4906 external_mini_cluster.cc:1428] Setting key 3d48279604f800dfb6b2bbed0f1910e5
Sep 26 06:24:42 dist-test-slave-pd0t krb5kdc[27518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403882, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.188@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
Sep 26 06:24:42 dist-test-slave-pd0t krb5kdc[27518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.188@KRBTEST.COM
I20260926 06:24:42.672407 27602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.663682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48556 (local address 127.4.202.190:38437)
0926 06:24:42.663757 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:42.663760 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:42.663932 (+ 172us) server_negotiation.cc:407] Connection header received
0926 06:24:42.664414 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:42.664415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:42.664427 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:42.664476 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:42.665070 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.665549 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.666660 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.666943 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.667040 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:42.669398 (+ 2358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:42.669400 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:42.669400 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:42.669407 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:42.670962 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:42.671426 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:42.671427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:42.671427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:42.671473 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:42.671752 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:42.671754 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:42.671754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:42.671931 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:42.672007 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:42.672236 (+ 229us) server_negotiation.cc:299] Negotiation successful
0926 06:24:42.672317 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 06:24:42.672537 27751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.663653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.190:38437 (local address 127.0.0.1:48556)
0926 06:24:42.663816 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:42.663848 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:42.664303 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:42.664555 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:42.664558 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:42.664778 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:42.664969 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:42.664973 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.665641 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.665642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:42.666548 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:42.666551 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.667063 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.667064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:42.667192 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:42.667665 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:24:42.667675 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:42.669312 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:42.671042 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:42.671049 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:42.671051 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:42.671349 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:42.671541 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:42.671542 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:42.671542 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:42.671673 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:42.672005 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:42.672009 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:42.672110 (+ 101us) client_negotiation.cc:769] Sending connection context
0926 06:24:42.672187 (+ 77us) client_negotiation.cc:240] Negotiation successful
0926 06:24:42.672336 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":74,"threads_started":1}
I20260926 06:24:42.673252 27749 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.202.189" port: 32903 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:24:42.675693 27753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.666780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41216 (local address 127.4.202.188:37851)
0926 06:24:42.667112 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:42.667116 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:42.667131 (+ 15us) server_negotiation.cc:407] Connection header received
0926 06:24:42.667148 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:42.667150 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:42.667165 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:42.667219 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:42.668007 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.668960 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.669801 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.670102 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.670181 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:42.672736 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:42.672742 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:42.672742 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:42.672749 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:42.674579 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:42.674940 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:42.674942 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:42.674943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:42.674994 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:42.675186 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:42.675187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:42.675187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:42.675301 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:42.675408 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:42.675523 (+ 115us) server_negotiation.cc:299] Negotiation successful
0926 06:24:42.675590 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":81,"threads_started":1}
Sep 26 06:24:42 dist-test-slave-pd0t krb5kdc[27518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403882, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.188@KRBTEST.COM for kudu/127.4.202.189@KRBTEST.COM
I20260926 06:24:42.680379 27679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.673579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46228 (local address 127.4.202.189:32903)
0926 06:24:42.673620 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:42.673621 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:42.673630 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:24:42.673656 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:42.673657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:42.673668 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:42.673721 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:42.673995 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.674758 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.675655 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.675945 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.676018 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:42.677900 (+ 1882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:42.677902 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:42.677902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:42.677910 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:42.679357 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:42.679687 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:42.679688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:42.679688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:42.679745 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:42.679946 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:42.679947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:42.679947 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:42.680054 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:42.680124 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:42.680263 (+ 139us) server_negotiation.cc:299] Negotiation successful
0926 06:24:42.680323 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 06:24:42.680409 27751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.673527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.189:32903 (local address 127.0.0.1:46228)
0926 06:24:42.673566 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:42.673575 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:42.673610 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:42.673724 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:42.673724 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:42.673885 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:42.673917 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:42.673918 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.674825 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.674826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:42.675579 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:42.675581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.676023 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.676024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:42.676067 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:42.676484 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:24:42.676489 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:42.677818 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:42.679437 (+ 1619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:42.679438 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:42.679438 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:42.679605 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:42.679815 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:42.679816 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:42.679816 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:42.679883 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:42.680127 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:42.680128 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:42.680168 (+ 40us) client_negotiation.cc:769] Sending connection context
0926 06:24:42.680236 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 06:24:42.680322 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":1}
Sep 26 06:24:42 dist-test-slave-pd0t krb5kdc[27518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.190@KRBTEST.COM
I20260926 06:24:42.681078 27749 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.202.188" port: 37851 } has no permanent_uuid. Determining permanent_uuid...
I20260926 06:24:42.683620 27602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.676468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48570 (local address 127.4.202.190:38437)
0926 06:24:42.676507 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:42.676508 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:42.676559 (+ 51us) server_negotiation.cc:407] Connection header received
0926 06:24:42.676629 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:42.676630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:42.676637 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:42.676679 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:42.677219 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.677868 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.678693 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.679017 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.679088 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:42.681146 (+ 2058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:42.681148 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:42.681148 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:42.681154 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:42.682520 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:42.682882 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:42.682883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:42.682883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:42.682941 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:42.683136 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:42.683137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:42.683138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:42.683242 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:42.683424 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:42.683460 (+ 36us) server_negotiation.cc:299] Negotiation successful
0926 06:24:42.683525 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Sep 26 06:24:42 dist-test-slave-pd0t krb5kdc[27518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403882, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.188@KRBTEST.COM for kudu/127.4.202.188@KRBTEST.COM
I20260926 06:24:42.687960 27751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.681440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:37851 (local address 127.0.0.1:41218)
0926 06:24:42.681476 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:42.681485 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:42.681517 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:42.681621 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:42.681622 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:42.681742 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:42.681764 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:42.681765 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.682369 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.682370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:42.682885 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:42.682886 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.683334 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.683334 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:42.683403 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:42.683916 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:24:42.683921 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:42.685073 (+ 1152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:42.686749 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:42.686750 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:42.686751 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:42.686996 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:42.687267 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:42.687267 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:42.687268 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:42.687345 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:42.687653 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:42.687653 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:42.687705 (+ 52us) client_negotiation.cc:769] Sending connection context
0926 06:24:42.687767 (+ 62us) client_negotiation.cc:240] Negotiation successful
0926 06:24:42.687857 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260926 06:24:42.688020 27753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.681496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41218 (local address 127.4.202.188:37851)
0926 06:24:42.681529 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:42.681531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:42.681538 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:24:42.681557 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:42.681558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:42.681567 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:42.681610 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:42.681839 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.682275 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.682975 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.683248 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.683333 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:42.685151 (+ 1818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:42.685152 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:42.685152 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:42.685157 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:42.686679 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:42.687087 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:42.687088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:42.687088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:42.687177 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:42.687421 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:42.687422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:42.687422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:42.687580 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:42.687660 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:42.687803 (+ 143us) server_negotiation.cc:299] Negotiation successful
0926 06:24:42.687903 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":19}
Sep 26 06:24:42 dist-test-slave-pd0t krb5kdc[27518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.202.189@KRBTEST.COM
I20260926 06:24:42.689404 27749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 252498b9bfc14e8cb61b0bd1acc4e7c9: Bootstrap starting.
I20260926 06:24:42.690347 27749 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 252498b9bfc14e8cb61b0bd1acc4e7c9: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:42.690718 27749 log.cc:824] T 00000000000000000000000000000000 P 252498b9bfc14e8cb61b0bd1acc4e7c9: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:42.690757 27599 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.202.188" port: 37851 } attempt: 2
I20260926 06:24:42.691614 27749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 252498b9bfc14e8cb61b0bd1acc4e7c9: No bootstrap required, opened a new log
I20260926 06:24:42.692353 27679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.684187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46236 (local address 127.4.202.189:32903)
0926 06:24:42.684231 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:42.684233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:42.684240 (+ 7us) server_negotiation.cc:407] Connection header received
0926 06:24:42.684261 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:42.684262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:42.684272 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:42.684331 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:42.685422 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.685824 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.686690 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.687091 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.687153 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:42.689584 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:42.689585 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:42.689585 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:42.689592 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:42.691242 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:42.691578 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:42.691580 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:42.691580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:42.691631 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:42.691869 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:42.691870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:42.691871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:42.691984 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:42.692139 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:42.692178 (+ 39us) server_negotiation.cc:299] Negotiation successful
0926 06:24:42.692244 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 06:24:42.693961 4906 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 06:24:42.693889 27749 raft_consensus.cc:348] T 00000000000000000000000000000000 P 252498b9bfc14e8cb61b0bd1acc4e7c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2871dc28296a4a0684c614f519e74abe" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 38437 } } peers { permanent_uuid: "939f962d1c244a8b92f48154f30e5562" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 32903 } } peers { permanent_uuid: "252498b9bfc14e8cb61b0bd1acc4e7c9" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 37851 } }
I20260926 06:24:42.694034 27749 raft_consensus.cc:729] T 00000000000000000000000000000000 P 252498b9bfc14e8cb61b0bd1acc4e7c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 252498b9bfc14e8cb61b0bd1acc4e7c9, State: Initialized, Role: FOLLOWER
I20260926 06:24:42.694130 27749 consensus_queue.cc:260] T 00000000000000000000000000000000 P 252498b9bfc14e8cb61b0bd1acc4e7c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2871dc28296a4a0684c614f519e74abe" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 38437 } } peers { permanent_uuid: "939f962d1c244a8b92f48154f30e5562" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 32903 } } peers { permanent_uuid: "252498b9bfc14e8cb61b0bd1acc4e7c9" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 37851 } }
I20260926 06:24:42.694428 27749 sys_catalog.cc:564] T 00000000000000000000000000000000 P 252498b9bfc14e8cb61b0bd1acc4e7c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:24:42.694381 27755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 252498b9bfc14e8cb61b0bd1acc4e7c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2871dc28296a4a0684c614f519e74abe" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 38437 } } peers { permanent_uuid: "939f962d1c244a8b92f48154f30e5562" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 32903 } } peers { permanent_uuid: "252498b9bfc14e8cb61b0bd1acc4e7c9" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 37851 } } }
I20260926 06:24:42.694489 27755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 252498b9bfc14e8cb61b0bd1acc4e7c9 [sys.catalog]: This master's current role is: FOLLOWER
Sep 26 06:24:42 dist-test-slave-pd0t krb5kdc[27518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403882, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.190@KRBTEST.COM for kudu/127.4.202.188@KRBTEST.COM
I20260926 06:24:42.694973 27573 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "2871dc28296a4a0684c614f519e74abe"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48570
W20260926 06:24:42.696141 27766 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 252498b9bfc14e8cb61b0bd1acc4e7c9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:24:42.696211 27766 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:24:42.698750 27753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.691154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41228 (local address 127.4.202.188:37851)
0926 06:24:42.691200 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:42.691202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:42.691210 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:24:42.691253 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:42.691253 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:42.691262 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:42.691317 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:42.691614 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.692268 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.693195 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.693491 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.693594 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:42.695845 (+ 2251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:42.695846 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:42.695846 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:42.695852 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:42.697427 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:42.697875 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:42.697877 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:42.697877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:42.697935 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:42.698149 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:42.698150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:42.698150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:42.698300 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:42.698381 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:42.698593 (+ 212us) server_negotiation.cc:299] Negotiation successful
0926 06:24:42.698668 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260926 06:24:42.698768 27601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.691110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:37851 (local address 127.0.0.1:41228)
0926 06:24:42.691154 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:42.691163 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:42.691201 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:42.691343 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:42.691344 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:42.691490 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:42.691527 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:42.691529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.692386 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.692387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:42.693098 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:42.693100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.693594 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.693595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:42.693663 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:42.694168 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:24:42.694173 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:42.695748 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:42.697516 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:42.697517 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:42.697518 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:42.697788 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:42.698012 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:42.698013 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:42.698013 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:42.698080 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:42.698414 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:42.698415 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:42.698464 (+ 49us) client_negotiation.cc:769] Sending connection context
0926 06:24:42.698537 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 06:24:42.698676 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260926 06:24:42.696619 27681 master_runner.cc:186] Error getting master registration for 127.4.202.190:38437: OK, instance_id { permanent_uuid: "2871dc28296a4a0684c614f519e74abe" instance_seqno: 1790403882350082 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 06:24:42.699648 27681 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 06:24:42.701098 27599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2871dc28296a4a0684c614f519e74abe: Bootstrap starting.
I20260926 06:24:42.701774 27599 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2871dc28296a4a0684c614f519e74abe: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:42.702042 27599 log.cc:824] T 00000000000000000000000000000000 P 2871dc28296a4a0684c614f519e74abe: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:42.702796 27599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2871dc28296a4a0684c614f519e74abe: No bootstrap required, opened a new log
I20260926 06:24:42.704100 27599 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2871dc28296a4a0684c614f519e74abe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2871dc28296a4a0684c614f519e74abe" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 38437 } } peers { permanent_uuid: "939f962d1c244a8b92f48154f30e5562" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 32903 } } peers { permanent_uuid: "252498b9bfc14e8cb61b0bd1acc4e7c9" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 37851 } }
I20260926 06:24:42.704232 27599 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2871dc28296a4a0684c614f519e74abe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2871dc28296a4a0684c614f519e74abe, State: Initialized, Role: FOLLOWER
I20260926 06:24:42.704312 27599 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2871dc28296a4a0684c614f519e74abe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2871dc28296a4a0684c614f519e74abe" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 38437 } } peers { permanent_uuid: "939f962d1c244a8b92f48154f30e5562" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 32903 } } peers { permanent_uuid: "252498b9bfc14e8cb61b0bd1acc4e7c9" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 37851 } }
I20260926 06:24:42.704610 27599 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2871dc28296a4a0684c614f519e74abe [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:24:42.704655 27573 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2871dc28296a4a0684c614f519e74abe [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260926 06:24:42.704694 27573 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2871dc28296a4a0684c614f519e74abe [term 0 FOLLOWER]: Advancing to term 1
I20260926 06:24:42.705499 27573 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2871dc28296a4a0684c614f519e74abe [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2871dc28296a4a0684c614f519e74abe" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 38437 } } peers { permanent_uuid: "939f962d1c244a8b92f48154f30e5562" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 32903 } } peers { permanent_uuid: "252498b9bfc14e8cb61b0bd1acc4e7c9" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 37851 } }
I20260926 06:24:42.705731 27573 leader_election.cc:288] T 00000000000000000000000000000000 P 2871dc28296a4a0684c614f519e74abe [CANDIDATE]: Term 1 election: Requested vote from peers 939f962d1c244a8b92f48154f30e5562 (127.4.202.189:32903), 252498b9bfc14e8cb61b0bd1acc4e7c9 (127.4.202.188:37851)
I20260926 06:24:42.704589 27769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2871dc28296a4a0684c614f519e74abe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2871dc28296a4a0684c614f519e74abe" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 38437 } } peers { permanent_uuid: "939f962d1c244a8b92f48154f30e5562" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 32903 } } peers { permanent_uuid: "252498b9bfc14e8cb61b0bd1acc4e7c9" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 37851 } } }
I20260926 06:24:42.706445 27769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2871dc28296a4a0684c614f519e74abe [sys.catalog]: This master's current role is: FOLLOWER
W20260926 06:24:42.706797 27780 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2871dc28296a4a0684c614f519e74abe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260926 06:24:42.706791 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2871dc28296a4a0684c614f519e74abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "252498b9bfc14e8cb61b0bd1acc4e7c9"
I20260926 06:24:42.706909 27723 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 252498b9bfc14e8cb61b0bd1acc4e7c9 [term 0 FOLLOWER]: Advancing to term 1
W20260926 06:24:42.706995 27780 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:24:42.707757 27723 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 252498b9bfc14e8cb61b0bd1acc4e7c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2871dc28296a4a0684c614f519e74abe in term 1.
I20260926 06:24:42.707998 27549 leader_election.cc:302] T 00000000000000000000000000000000 P 2871dc28296a4a0684c614f519e74abe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 252498b9bfc14e8cb61b0bd1acc4e7c9, 2871dc28296a4a0684c614f519e74abe; no voters:
I20260926 06:24:42.708112 27769 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2871dc28296a4a0684c614f519e74abe [term 1 FOLLOWER]: Leader election won for term 1
I20260926 06:24:42.708236 27769 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2871dc28296a4a0684c614f519e74abe [term 1 LEADER]: Becoming Leader. State: Replica: 2871dc28296a4a0684c614f519e74abe, State: Running, Role: LEADER
I20260926 06:24:42.708326 27769 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2871dc28296a4a0684c614f519e74abe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2871dc28296a4a0684c614f519e74abe" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 38437 } } peers { permanent_uuid: "939f962d1c244a8b92f48154f30e5562" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 32903 } } peers { permanent_uuid: "252498b9bfc14e8cb61b0bd1acc4e7c9" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 37851 } }
I20260926 06:24:42.708683 27769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2871dc28296a4a0684c614f519e74abe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2871dc28296a4a0684c614f519e74abe. Latest consensus state: current_term: 1 leader_uuid: "2871dc28296a4a0684c614f519e74abe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2871dc28296a4a0684c614f519e74abe" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 38437 } } peers { permanent_uuid: "939f962d1c244a8b92f48154f30e5562" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 32903 } } peers { permanent_uuid: "252498b9bfc14e8cb61b0bd1acc4e7c9" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 37851 } } }
I20260926 06:24:42.708755 27769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2871dc28296a4a0684c614f519e74abe [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:42.709537 27791 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 06:24:42.709815 27791 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 06:24:42.709681 27531 master_runner.cc:186] Error getting master registration for 127.4.202.189:32903: OK, instance_id { permanent_uuid: "939f962d1c244a8b92f48154f30e5562" instance_seqno: 1790403882513090 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 06:24:42.712875 27531 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 06:24:42.713666 27723 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 252498b9bfc14e8cb61b0bd1acc4e7c9 [term 1 FOLLOWER]: Refusing update from remote peer 2871dc28296a4a0684c614f519e74abe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:24:42.713936 27769 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2871dc28296a4a0684c614f519e74abe [LEADER]: Connected to new peer: Peer: permanent_uuid: "252498b9bfc14e8cb61b0bd1acc4e7c9" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 37851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:24:42.715061 27755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 252498b9bfc14e8cb61b0bd1acc4e7c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2871dc28296a4a0684c614f519e74abe. Latest consensus state: current_term: 1 leader_uuid: "2871dc28296a4a0684c614f519e74abe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2871dc28296a4a0684c614f519e74abe" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 38437 } } peers { permanent_uuid: "939f962d1c244a8b92f48154f30e5562" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 32903 } } peers { permanent_uuid: "252498b9bfc14e8cb61b0bd1acc4e7c9" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 37851 } } }
I20260926 06:24:42.715153 27755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 252498b9bfc14e8cb61b0bd1acc4e7c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:24:42.716969 27602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.706982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48572 (local address 127.4.202.190:38437)
0926 06:24:42.707025 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:42.707027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:42.707880 (+ 853us) server_negotiation.cc:407] Connection header received
0926 06:24:42.707988 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:42.707990 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:42.708002 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:42.708036 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:42.708818 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.709495 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.710485 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.710781 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.710821 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:42.712382 (+ 1561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:42.712383 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:42.712384 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:42.712390 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:42.715845 (+ 3455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:42.716218 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:42.716219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:42.716220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:42.716287 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:42.716488 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:42.716489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:42.716490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:42.716607 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:42.716694 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:42.716814 (+ 120us) server_negotiation.cc:299] Negotiation successful
0926 06:24:42.716886 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 06:24:42.724974 27679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.707288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46242 (local address 127.4.202.189:32903)
0926 06:24:42.707326 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:42.707328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:42.717144 (+ 9816us) server_negotiation.cc:407] Connection header received
0926 06:24:42.717181 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:42.717182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:42.717193 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:42.717237 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:42.718043 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.718802 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.719772 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.720510 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.720582 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:42.721883 (+ 1301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:42.721885 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:42.721885 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:42.721892 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:42.723571 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:42.723989 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:42.723990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:42.723991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:42.724055 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:42.724335 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:42.724336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:42.724336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:42.724458 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:42.724541 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:42.724794 (+ 253us) server_negotiation.cc:299] Negotiation successful
0926 06:24:42.724882 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 06:24:42.727883 27755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 252498b9bfc14e8cb61b0bd1acc4e7c9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2871dc28296a4a0684c614f519e74abe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2871dc28296a4a0684c614f519e74abe" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 38437 } } peers { permanent_uuid: "939f962d1c244a8b92f48154f30e5562" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 32903 } } peers { permanent_uuid: "252498b9bfc14e8cb61b0bd1acc4e7c9" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 37851 } } }
I20260926 06:24:42.727972 27755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 252498b9bfc14e8cb61b0bd1acc4e7c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:24:42.728148 27769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2871dc28296a4a0684c614f519e74abe [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2871dc28296a4a0684c614f519e74abe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2871dc28296a4a0684c614f519e74abe" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 38437 } } peers { permanent_uuid: "939f962d1c244a8b92f48154f30e5562" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 32903 } } peers { permanent_uuid: "252498b9bfc14e8cb61b0bd1acc4e7c9" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 37851 } } }
I20260926 06:24:42.728233 27769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2871dc28296a4a0684c614f519e74abe [sys.catalog]: This master's current role is: LEADER
I20260926 06:24:42.729297 27793 mvcc.cc:205] Tried to move back new op lower bound from 7333494303592595456 to 7333494303573876736. Current Snapshot: MvccSnapshot[applied={T|T < 7333494303592595456}]
I20260926 06:24:42.731045 27791 catalog_manager.cc:1347] Generated new cluster ID: 044f672cc4864de584424343597e5f7a
I20260926 06:24:42.731089 27791 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 06:24:42.731562 27753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.707577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41238 (local address 127.4.202.188:37851)
0926 06:24:42.707615 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:42.707616 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:42.718692 (+ 11076us) server_negotiation.cc:407] Connection header received
0926 06:24:42.718752 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:42.718753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:42.718764 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:42.718822 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:42.719463 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.720112 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.721243 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.721545 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.721621 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:42.728331 (+ 6710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:42.728332 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:42.728332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:42.728339 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:42.730038 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:42.730466 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:42.730467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:42.730467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:42.730521 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:42.730782 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:42.730783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:42.730784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:42.730914 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:42.731009 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:42.731404 (+ 395us) server_negotiation.cc:299] Negotiation successful
0926 06:24:42.731476 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 06:24:42.748271 27791 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 06:24:42.748454 27674 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.202.188" port: 37851 } attempt: 3
I20260926 06:24:42.748978 27791 catalog_manager.cc:1504] Loading token signing keys...
Sep 26 06:24:42 dist-test-slave-pd0t krb5kdc[27518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790403882, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.202.189@KRBTEST.COM for kudu/127.4.202.188@KRBTEST.COM
I20260926 06:24:42.756223 27676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.748664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.202.188:37851 (local address 127.0.0.1:41246)
0926 06:24:42.748705 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0926 06:24:42.748716 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 06:24:42.748762 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 06:24:42.748929 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 06:24:42.748930 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 06:24:42.749094 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0926 06:24:42.749138 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:42.749140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.750023 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.750024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:42.750852 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 06:24:42.750855 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.751347 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.751349 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 06:24:42.751405 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:42.751885 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 06:24:42.751890 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 06:24:42.753378 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 06:24:42.755218 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:42.755219 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:42.755220 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:42.755433 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:42.755628 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 06:24:42.755628 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 06:24:42.755629 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 06:24:42.755682 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 06:24:42.755942 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 06:24:42.755942 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 06:24:42.755983 (+ 41us) client_negotiation.cc:769] Sending connection context
0926 06:24:42.756043 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 06:24:42.756129 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260926 06:24:42.756798 27753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:42.748789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41246 (local address 127.4.202.188:37851)
0926 06:24:42.748822 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:42.748824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:42.748832 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:24:42.748851 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:42.748851 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:42.748860 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:42.748918 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:42.749220 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.749931 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.750942 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:42.751233 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:42.751289 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:42.753456 (+ 2167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 06:24:42.753457 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 06:24:42.753457 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 06:24:42.753464 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 06:24:42.755126 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:42.755510 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:42.755511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:42.755511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:42.755562 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 06:24:42.755743 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 06:24:42.755743 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 06:24:42.755744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 06:24:42.755855 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 06:24:42.755916 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:42.756627 (+ 711us) server_negotiation.cc:299] Negotiation successful
0926 06:24:42.756715 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 06:24:42.758447 27674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 939f962d1c244a8b92f48154f30e5562: Bootstrap starting.
I20260926 06:24:42.759148 27674 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 939f962d1c244a8b92f48154f30e5562: Neither blocks nor log segments found. Creating new log.
I20260926 06:24:42.759438 27674 log.cc:824] T 00000000000000000000000000000000 P 939f962d1c244a8b92f48154f30e5562: Log is configured to *not* fsync() on all Append() calls
I20260926 06:24:42.760434 27674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 939f962d1c244a8b92f48154f30e5562: No bootstrap required, opened a new log
I20260926 06:24:42.762459 27674 raft_consensus.cc:348] T 00000000000000000000000000000000 P 939f962d1c244a8b92f48154f30e5562 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2871dc28296a4a0684c614f519e74abe" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 38437 } } peers { permanent_uuid: "939f962d1c244a8b92f48154f30e5562" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 32903 } } peers { permanent_uuid: "252498b9bfc14e8cb61b0bd1acc4e7c9" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 37851 } }
I20260926 06:24:42.762627 27674 raft_consensus.cc:729] T 00000000000000000000000000000000 P 939f962d1c244a8b92f48154f30e5562 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 939f962d1c244a8b92f48154f30e5562, State: Initialized, Role: FOLLOWER
I20260926 06:24:42.762704 27674 consensus_queue.cc:260] T 00000000000000000000000000000000 P 939f962d1c244a8b92f48154f30e5562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2871dc28296a4a0684c614f519e74abe" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 38437 } } peers { permanent_uuid: "939f962d1c244a8b92f48154f30e5562" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 32903 } } peers { permanent_uuid: "252498b9bfc14e8cb61b0bd1acc4e7c9" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 37851 } }
I20260926 06:24:42.763025 27674 sys_catalog.cc:564] T 00000000000000000000000000000000 P 939f962d1c244a8b92f48154f30e5562 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 06:24:42.763309 27791 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2871dc28296a4a0684c614f519e74abe: Generated new TSK 0
I20260926 06:24:42.763466 27791 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 06:24:42.764309 27647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 939f962d1c244a8b92f48154f30e5562 [term 0 FOLLOWER]: Advancing to term 1
W20260926 06:24:42.764703 27810 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 939f962d1c244a8b92f48154f30e5562: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 06:24:42.764748 27810 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 06:24:42.765161 27647 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 939f962d1c244a8b92f48154f30e5562 [term 1 FOLLOWER]: Refusing update from remote peer 2871dc28296a4a0684c614f519e74abe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 06:24:42.765380 27648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2871dc28296a4a0684c614f519e74abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "939f962d1c244a8b92f48154f30e5562"
I20260926 06:24:42.766099 27648 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 939f962d1c244a8b92f48154f30e5562 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2871dc28296a4a0684c614f519e74abe in term 1.
I20260926 06:24:42.766928 27789 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2871dc28296a4a0684c614f519e74abe [LEADER]: Connected to new peer: Peer: permanent_uuid: "939f962d1c244a8b92f48154f30e5562" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 32903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 06:24:42.769150 27815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 939f962d1c244a8b92f48154f30e5562 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2871dc28296a4a0684c614f519e74abe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2871dc28296a4a0684c614f519e74abe" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 38437 } } peers { permanent_uuid: "939f962d1c244a8b92f48154f30e5562" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 32903 } } peers { permanent_uuid: "252498b9bfc14e8cb61b0bd1acc4e7c9" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 37851 } } }
I20260926 06:24:42.769286 27815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 939f962d1c244a8b92f48154f30e5562 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:24:42.769398 27815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 939f962d1c244a8b92f48154f30e5562 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2871dc28296a4a0684c614f519e74abe. Latest consensus state: current_term: 1 leader_uuid: "2871dc28296a4a0684c614f519e74abe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2871dc28296a4a0684c614f519e74abe" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 38437 } } peers { permanent_uuid: "939f962d1c244a8b92f48154f30e5562" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 32903 } } peers { permanent_uuid: "252498b9bfc14e8cb61b0bd1acc4e7c9" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 37851 } } }
I20260926 06:24:42.769519 27815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 939f962d1c244a8b92f48154f30e5562 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:24:42.769593 27815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 939f962d1c244a8b92f48154f30e5562 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2871dc28296a4a0684c614f519e74abe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2871dc28296a4a0684c614f519e74abe" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 38437 } } peers { permanent_uuid: "939f962d1c244a8b92f48154f30e5562" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 32903 } } peers { permanent_uuid: "252498b9bfc14e8cb61b0bd1acc4e7c9" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 37851 } } }
I20260926 06:24:42.769696 27815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 939f962d1c244a8b92f48154f30e5562 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 06:24:42.770411 27813 mvcc.cc:205] Tried to move back new op lower bound from 7333494303793434624 to 7333494303573876736. Current Snapshot: MvccSnapshot[applied={T|T < 7333494303592595456}]
W20260926 06:24:42.789812 27602 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:24:42.788373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48582 (local address 127.4.202.190:38437)
0926 06:24:42.788413 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:42.788415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:42.789220 (+ 805us) server_negotiation.cc:407] Connection header received
0926 06:24:42.789315 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:42.789316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:42.789327 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:42.789356 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:42.789752 (+ 396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48582: BlockingRecv error: recv got EOF from 127.0.0.1:48582 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260926 06:24:42.790482 27679 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:24:42.788701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46248 (local address 127.4.202.189:32903)
0926 06:24:42.788737 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:42.788738 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:42.790034 (+ 1296us) server_negotiation.cc:407] Connection header received
0926 06:24:42.790122 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:42.790123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:42.790133 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:42.790162 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:42.790431 (+ 269us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46248: BlockingRecv error: recv got EOF from 127.0.0.1:46248 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260926 06:24:42.791191 27753 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:24:42.788956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41262 (local address 127.4.202.188:37851)
0926 06:24:42.788989 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:42.788991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:42.790699 (+ 1708us) server_negotiation.cc:407] Connection header received
0926 06:24:42.790800 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:42.790801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:42.790812 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:42.790839 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:42.791137 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41262: BlockingRecv error: recv got EOF from 127.0.0.1:41262 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260926 06:24:42.792956 27679 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:24:42.792246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46254 (local address 127.4.202.189:32903)
0926 06:24:42.792286 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:42.792288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:42.792499 (+ 211us) server_negotiation.cc:407] Connection header received
0926 06:24:42.792594 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:42.792595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:42.792610 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:42.792638 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:42.792903 (+ 265us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46254: BlockingRecv error: recv got EOF from 127.0.0.1:46254 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260926 06:24:42.795749 27679 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:24:42.794990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46268 (local address 127.4.202.189:32903)
0926 06:24:42.795020 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:42.795022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:42.795304 (+ 282us) server_negotiation.cc:407] Connection header received
0926 06:24:42.795398 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:42.795399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:42.795407 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:42.795435 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:42.795712 (+ 277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46268: BlockingRecv error: recv got EOF from 127.0.0.1:46268 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260926 06:24:42.799654 27679 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:24:42.798931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46278 (local address 127.4.202.189:32903)
0926 06:24:42.798974 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:42.798975 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:42.799209 (+ 234us) server_negotiation.cc:407] Connection header received
0926 06:24:42.799297 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:42.799298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:42.799307 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:42.799334 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:42.799613 (+ 279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46278: BlockingRecv error: recv got EOF from 127.0.0.1:46278 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260926 06:24:42.805418 27679 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:24:42.804669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46280 (local address 127.4.202.189:32903)
0926 06:24:42.804699 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:42.804701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:42.804906 (+ 205us) server_negotiation.cc:407] Connection header received
0926 06:24:42.805011 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:42.805012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:42.805020 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:42.805047 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:42.805380 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46280: BlockingRecv error: recv got EOF from 127.0.0.1:46280 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260926 06:24:42.815202 27679 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:24:42.814453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46296 (local address 127.4.202.189:32903)
0926 06:24:42.814486 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:42.814488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:42.814730 (+ 242us) server_negotiation.cc:407] Connection header received
0926 06:24:42.814820 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:42.814820 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:42.814831 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:42.814858 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:42.815161 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46296: BlockingRecv error: recv got EOF from 127.0.0.1:46296 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260926 06:24:42.833087 27679 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:24:42.832287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46302 (local address 127.4.202.189:32903)
0926 06:24:42.832321 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:42.832323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:42.832574 (+ 251us) server_negotiation.cc:407] Connection header received
0926 06:24:42.832667 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:42.832669 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:42.832679 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:42.832707 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:42.833041 (+ 334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46302: BlockingRecv error: recv got EOF from 127.0.0.1:46302 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260926 06:24:42.867388 27679 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:24:42.866407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46304 (local address 127.4.202.189:32903)
0926 06:24:42.866452 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:42.866454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:42.866758 (+ 304us) server_negotiation.cc:407] Connection header received
0926 06:24:42.866865 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:42.866866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:42.866878 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:42.866919 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:42.867327 (+ 408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46304: BlockingRecv error: recv got EOF from 127.0.0.1:46304 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260926 06:24:42.933432 27679 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:24:42.932691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46320 (local address 127.4.202.189:32903)
0926 06:24:42.932737 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:42.932739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:42.932860 (+ 121us) server_negotiation.cc:407] Connection header received
0926 06:24:42.932962 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:42.932963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:42.932974 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:42.933041 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:42.933368 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46320: BlockingRecv error: recv got EOF from 127.0.0.1:46320 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260926 06:24:43.063464 27679 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:24:43.062660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46332 (local address 127.4.202.189:32903)
0926 06:24:43.062721 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:43.062723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:43.062832 (+ 109us) server_negotiation.cc:407] Connection header received
0926 06:24:43.062938 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:43.062939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:43.062952 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:43.063025 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:43.063369 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46332: BlockingRecv error: recv got EOF from 127.0.0.1:46332 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260926 06:24:43.270723 27789 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2871dc28296a4a0684c614f519e74abe [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2871dc28296a4a0684c614f519e74abe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2871dc28296a4a0684c614f519e74abe" member_type: VOTER last_known_addr { host: "127.4.202.190" port: 38437 } } peers { permanent_uuid: "939f962d1c244a8b92f48154f30e5562" member_type: VOTER last_known_addr { host: "127.4.202.189" port: 32903 } } peers { permanent_uuid: "252498b9bfc14e8cb61b0bd1acc4e7c9" member_type: VOTER last_known_addr { host: "127.4.202.188" port: 37851 } } }
I20260926 06:24:43.270934 27789 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2871dc28296a4a0684c614f519e74abe [sys.catalog]: This master's current role is: LEADER
W20260926 06:24:43.321415 27679 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 06:24:43.320737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46340 (local address 127.4.202.189:32903)
0926 06:24:43.320805 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:43.320807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:43.320850 (+ 43us) server_negotiation.cc:407] Connection header received
0926 06:24:43.320883 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:43.320884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:43.320896 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:43.320974 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 06:24:43.321350 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46340: BlockingRecv error: recv got EOF from 127.0.0.1:46340 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 06:24:43.696594 27766 catalog_manager.cc:1259] Loaded cluster ID: 044f672cc4864de584424343597e5f7a
I20260926 06:24:43.696663 27766 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 252498b9bfc14e8cb61b0bd1acc4e7c9: loading cluster ID for follower catalog manager: success
I20260926 06:24:43.697543 27766 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 252498b9bfc14e8cb61b0bd1acc4e7c9: acquiring CA information for follower catalog manager: success
I20260926 06:24:43.697873 27766 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 252498b9bfc14e8cb61b0bd1acc4e7c9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 06:24:43.765133 27810 catalog_manager.cc:1259] Loaded cluster ID: 044f672cc4864de584424343597e5f7a
I20260926 06:24:43.765240 27810 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 939f962d1c244a8b92f48154f30e5562: loading cluster ID for follower catalog manager: success
I20260926 06:24:43.766058 27810 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 939f962d1c244a8b92f48154f30e5562: acquiring CA information for follower catalog manager: success
I20260926 06:24:43.766390 27810 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 939f962d1c244a8b92f48154f30e5562: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 06:24:43.837195 27885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:43.834507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46348 (local address 127.4.202.189:32903)
0926 06:24:43.834652 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:43.834654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:43.834663 (+ 9us) server_negotiation.cc:407] Connection header received
0926 06:24:43.834719 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:43.834720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:43.834762 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:43.834856 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:24:43.834997 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:43.835517 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:43.836293 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:43.836490 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:43.836554 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:43.836739 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:24:43.836873 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:24:43.836950 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:43.837045 (+ 95us) server_negotiation.cc:299] Negotiation successful
0926 06:24:43.837067 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":55,"threads_started":1}
I20260926 06:24:43.841041 27891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:43.838971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41264 (local address 127.4.202.188:37851)
0926 06:24:43.839074 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:43.839076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:43.839084 (+ 8us) server_negotiation.cc:407] Connection header received
0926 06:24:43.839135 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:43.839136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:43.839168 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:43.839234 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:24:43.839305 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:43.839755 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:43.840432 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:43.840614 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:43.840664 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:43.840785 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:24:43.840878 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:24:43.840932 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:43.840968 (+ 36us) server_negotiation.cc:299] Negotiation successful
0926 06:24:43.840986 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":60,"threads_started":1}
I20260926 06:24:43.845799 27897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:43.842678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48586 (local address 127.4.202.190:38437)
0926 06:24:43.842781 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:43.842783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:43.842793 (+ 10us) server_negotiation.cc:407] Connection header received
0926 06:24:43.842894 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:43.842895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:43.842947 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:43.843026 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:24:43.843214 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:43.843798 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:43.844801 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:43.845147 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:43.845230 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:43.845425 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:24:43.845521 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:24:43.845577 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:43.845690 (+ 113us) server_negotiation.cc:299] Negotiation successful
0926 06:24:43.845720 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":51,"threads_started":1}
I20260926 06:24:43.845809 27891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:43.843130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41276 (local address 127.4.202.188:37851)
0926 06:24:43.843172 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:43.843174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:43.843200 (+ 26us) server_negotiation.cc:407] Connection header received
0926 06:24:43.843258 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:43.843259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:43.843268 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:43.843322 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:24:43.843447 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:43.844136 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:43.845065 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:43.845370 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:43.845439 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:43.845555 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:24:43.845584 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:24:43.845628 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:43.845722 (+ 94us) server_negotiation.cc:299] Negotiation successful
0926 06:24:43.845741 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 06:24:43.845948 27885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 06:24:43.842943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46364 (local address 127.4.202.189:32903)
0926 06:24:43.842984 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 06:24:43.842986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 06:24:43.843032 (+ 46us) server_negotiation.cc:407] Connection header received
0926 06:24:43.843130 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 06:24:43.843131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 06:24:43.843142 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 06:24:43.843171 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 06:24:43.843374 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:43.844099 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:43.845115 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 06:24:43.845414 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 06:24:43.845469 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 06:24:43.845653 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 06:24:43.845690 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 06:24:43.845737 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 06:24:43.845838 (+ 101us) server_negotiation.cc:299] Negotiation successful
0926 06:24:43.845860 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 06:24:43.852607 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 27531
I20260926 06:24:43.857924 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 27606
I20260926 06:24:43.862679 4906 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm7eEJS/build/release/bin/kudu with pid 27681
2026-09-26T06:24:43Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2768 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5577 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (292229 ms total)
[ PASSED ] 54 tests.
[ FAILED ] 2 tests, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
[ FAILED ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
2 FAILED TESTS